Envoy Payload-To-Metadata Filter
This filter should be configured with the type URL
type.googleapis.com/envoy.extensions.filters.network.thrift_proxy.filters.payload_to_metadata.v3.PayloadToMetadata.
A typical use case for this filter is to dynamically match a specified payload field of requests with load balancer subsets. For this, a given payload field’s value would be extracted and attached to the request as dynamic metadata which would then be used to match a subset of endpoints.
We already have header-To-metadata filter to achieve the similar goal. However, we have two reasons for introducing new payload-To-metadata filter:
1. Transports like framed transport don’t support THeaders, which is unable to use Header-To-Metadata filter.
2. Directly referring to payload field stops envoy relying on that the downstream service always copies the field to the THeader correctly and guarantees single truth of source.
This filter is configured with request_rules that will be matched against requests. A field_selector of a rule represents the head of a linked list, each node of the linked list has a name for logging and an id for matching. The field_selector is tied to a payload field when the linked list corresponds to a downward path which rooted in the top-level of the request message structure. on_present is triggered when corresponding the payload is present. Otherwise, on_missing is triggered.
Note that if the corresponding payload for a rule is present but on_present is missing, no metadata is added for this rule. . If the corresponding payload for a rule is an empty string, neither on_present nor on_missing is triggered. i.e., no metadata is added for this rule.
Currently payload to metadata filter doesn’t support container type payload, i.e., list, set and map.
We limit the size of a single metadata value which is added by this filter to 1024 bytes.
This filter is designed to support payload passthrough. Performing payload to metadata filter can do deserialization once, and pass the metadata to other filters. This means that load balancing decisions, consumed from log and routing could all use payload information with a single parse. Also notably performing the parsing in payload passthrough buffer will mean deserialization once and not re-serializing, which is the most performant outcome. Currently there is a redundant buffer copy until we have BufferView.
If any of the filter chain doesn’t support payload passthrough, a customized non-passthrough filter to setup metadata is encouraged from point of performance view.
Example
A sample filter configuration to route traffic to endpoints based on the presence or absence of a version payload could be:
20 thrift_filters:
21 - name: envoy.filters.thrift.payload_to_metadata
22 typed_config:
23 "@type": type.googleapis.com/envoy.extensions.filters.network.thrift_proxy.filters.payload_to_metadata.v3.PayloadToMetadata
24 request_rules:
25 - method_name: foo
26 field_selector:
27 name: info
28 id: 2
29 child:
30 name: version
31 id: 1
32 on_present:
33 metadata_namespace: envoy.lb
34 key: version
35 on_missing:
36 metadata_namespace: envoy.lb
37 key: default
38 value: 'unknown'
A corresponding upstream cluster configuration could be:
37 type: STRICT_DNS
38 lb_policy: ROUND_ROBIN
39 lb_subset_config:
40 fallback_policy: NO_FALLBACK
41 subset_selectors:
42 - keys:
43 - default
44 - keys:
45 - version
46 load_assignment:
47 cluster_name: versioned-cluster
The request thrift structure could be:
syntax = "proto3";
package request;
message Request {
message Info {
string version = 1;
}
string data = 1;
Info info = 2;
}
This would then allow requests of method name foo with the version payload field which is
under info field set to be matched against endpoints with the corresponding version. Whereas
requests with that payload missing would be matched with the default endpoints.
The regex matching and substitution is similar with header to metadata filter.
Statistics
Currently, this filter generates no statistics.