Envoy Payload-To-Metadata Filter

  • This filter should be configured with the type URL type.googleapis.com/envoy.extensions.filters.network.thrift_proxy.filters.payload_to_metadata.v3.PayloadToMetadata.

  • v3 API reference

A typical use case for this filter is to dynamically match a specified payload field of requests with load balancer subsets. For this, a given payload field’s value would be extracted and attached to the request as dynamic metadata which would then be used to match a subset of endpoints.

We already have header-To-metadata filter to achieve the similar goal. However, we have two reasons for introducing new payload-To-metadata filter:

1. Transports like framed transport don’t support THeaders, which is unable to use Header-To-Metadata filter.

2. Directly referring to payload field stops envoy relying on that the downstream service always copies the field to the THeader correctly and guarantees single truth of source.

This filter is configured with request_rules that will be matched against requests. A field_selector of a rule represents the head of a linked list, each node of the linked list has a name for logging and an id for matching. The field_selector is tied to a payload field when the linked list corresponds to a downward path which rooted in the top-level of the request message structure. on_present is triggered when corresponding the payload is present. Otherwise, on_missing is triggered.

Note that if the corresponding payload for a rule is present but on_present is missing, no metadata is added for this rule. . If the corresponding payload for a rule is an empty string, neither on_present nor on_missing is triggered. i.e., no metadata is added for this rule.

Currently payload to metadata filter doesn’t support container type payload, i.e., list, set and map.

We limit the size of a single metadata value which is added by this filter to 1024 bytes.

This filter is designed to support payload passthrough. Performing payload to metadata filter can do deserialization once, and pass the metadata to other filters. This means that load balancing decisions, consumed from log and routing could all use payload information with a single parse. Also notably performing the parsing in payload passthrough buffer will mean deserialization once and not re-serializing, which is the most performant outcome. Currently there is a redundant buffer copy until we have BufferView.

If any of the filter chain doesn’t support payload passthrough, a customized non-passthrough filter to setup metadata is encouraged from point of performance view.


A sample filter configuration to route traffic to endpoints based on the presence or absence of a version payload could be:

20          thrift_filters:
21          - name: envoy.filters.thrift.payload_to_metadata
22            typed_config:
23              "@type": type.googleapis.com/envoy.extensions.filters.network.thrift_proxy.filters.payload_to_metadata.v3.PayloadToMetadata
24              request_rules:
25              - method_name: foo
26                field_selector:
27                  name: info
28                  id: 2
29                  child:
30                    name: version
31                    id: 1
32                on_present:
33                  metadata_namespace: envoy.lb
34                  key: version
35                on_missing:
36                  metadata_namespace: envoy.lb
37                  key: default
38                  value: 'unknown'

A corresponding upstream cluster configuration could be:

37    type: STRICT_DNS
38    lb_policy: ROUND_ROBIN
39    lb_subset_config:
40      fallback_policy: NO_FALLBACK
41      subset_selectors:
42      - keys:
43        - default
44      - keys:
45        - version
46    load_assignment:
47      cluster_name: versioned-cluster

The request thrift structure could be:

syntax = "proto3";

package request;

message Request {
  message Info {
    string version = 1;
  string data = 1;
  Info info = 2;

This would then allow requests of method name foo with the version payload field which is under info field set to be matched against endpoints with the corresponding version. Whereas requests with that payload missing would be matched with the default endpoints.

The regex matching and substitution is similar with header to metadata filter.


Currently, this filter generates no statistics.