gRPC Field Extraction (proto)
This extension has the qualified name envoy.filters.http.grpc_field_extraction
Note
This extension is functional but has not had substantial production burn time, use only with this caveat.
This extension has an unknown security posture and should only be used in deployments where both the downstream and upstream are trusted.
Tip
This extension extends and can be used with the following extension category:
This extension must be configured with one of the following type URLs:
Overview
This filter supports extracting the fields from the first gRPC request message no matter if it is unary or streaming and writing the result to the destination, for which currently only the static Envoy dynamic metadata envoy.filters.http.grpc_field_extraction is supported.
Assumptions
This filter assumes
this filter is only applicable for gRPC with Protobuf as payload.
for bi-directional and client-side gRPC streaming, the initial message from the client should not depend on receiving the server initial metadata.
Process Flow
When a request reaches the filter, it will check
if the request is the gRPC request configured for extraction, the filter tries to:
block the incoming data before decoding the first complete gRPC message
look up the target field from the buffered gRPC message
write the extraction result into the dynamic metadata and resume the request propagation.
otherwise, pass through the request.
If the request is a malformed one found during 1.a or 1.b, the filter will reject the request.
Config Requirements
Here are config requirements
the target field should be among the following primitive types: string, uint32, uint64, int32, int64, sint32, sint64, fixed32, fixed64, sfixed32, sfixed64, float, double.
the target field could be repeated.
the intermediate type could also be repeated.
Output Format
the extracted field names/values will be wrapped in be
field<StringValue>
->values<ListValue of StringValue>
, which will be added in the dynamicmetadata<google.protobuf.Struct>
.if the field value is empty, a empty
<ListValue>
will be set.
Performance
This filter should be performant as it
converts between the gRPC message from EnvoyBuffer without data copy.
parse the gRPC message binary directly without deserialization.
though buffering the first message introduces some latency.
Example
We have the following request definition for the gRPC method pkg.svc.Method.
message MethodRequest {
string foo = 1;
Nested nested = 2;
uint32 baz = 3;
...
}
message Nested {
repeated string bar = 1;
}
This is the filter config in JSON.
{
"descriptor_set":{},
"extractions_by_method":{
"pkg.svc.Method":{
"request_field_extractions":{
"foo":{
},
"nested.bar":{
}
"baz":{
}
}
}
}
}
During runtime, the filter receives the following MethodRequest message in JSON.
{
"foo": "val_foo",
"nested": { "bar": ["val_bar1", "val_bar2"]}
}
The filter will write the following dynamic metadata(envoy.filters.http.grpc_field_extraction) in JSON.
{
"foo":[
"val_foo"
],
"nested.bar":[
"val_bar1", "val_bar2"
]
"baz":[
]
}
extensions.filters.http.grpc_field_extraction.v3.GrpcFieldExtractionConfig
[extensions.filters.http.grpc_field_extraction.v3.GrpcFieldExtractionConfig proto]
{
"descriptor_set": {...},
"extractions_by_method": {...}
}
- descriptor_set
(config.core.v3.DataSource, REQUIRED) The proto descriptor set binary for the gRPC services.
It could be passed by a local file through
Datasource.filename
or embedded in theDatasource.inline_bytes
.
- extractions_by_method
(repeated map<string, extensions.filters.http.grpc_field_extraction.v3.FieldExtractions>) Specify the extraction info. The key is the fully qualified gRPC method name.
${package}.${Service}.${Method}
, likeendpoints.examples.bookstore.BookStore.GetShelf
The value is the field extractions for individual gRPC method.
extensions.filters.http.grpc_field_extraction.v3.FieldExtractions
[extensions.filters.http.grpc_field_extraction.v3.FieldExtractions proto]
This message can be used to support per route config approach later even though the Istio doesn’t support that so far.
{
"request_field_extractions": {...}
}
- request_field_extractions
(repeated map<string, extensions.filters.http.grpc_field_extraction.v3.RequestFieldValueDisposition>) The field extractions for requests. The key is the field path within the grpc request. For example, we can define
foo.bar.name
if we want to extractRequest.foo.bar.name
.message Request { Foo foo = 1; } message Foo { Bar bar = 1; } message Bar { string name = 1; }
extensions.filters.http.grpc_field_extraction.v3.RequestFieldValueDisposition
[extensions.filters.http.grpc_field_extraction.v3.RequestFieldValueDisposition proto]
{
"dynamic_metadata": ...
}
- dynamic_metadata
(string) The dynamic metadata namespace. If empty, “envoy.filters.http.grpc_field_extraction” will be used by default.
Unimplemented. Uses “envoy.filters.http.grpc_field_extraction” for now.