UDP proxy

  • This filter should be configured with the type URL type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.UdpProxyConfig.

  • v3 API reference

Overview

The UDP proxy listener filter allows Envoy to operate as a non-transparent proxy between a UDP client and server. The lack of transparency means that the upstream server will see the source IP and port of the Envoy instance versus the client. All datagrams flow from the client, to Envoy, to the upstream server, back to Envoy, and back to the client.

Because UDP is not a connection oriented protocol, Envoy must keep track of a client’s session such that the response datagrams from an upstream server can be routed back to the correct client. Each session is index by the 4-tuple consisting of source IP/port and local IP/port that the datagram is received on. Sessions last until the idle timeout is reached.

Above session stickness could be disabled by setting use_per_packet_load_balancing to true. In that case, per packet load balancing is enabled. It means that upstream host is selected on every single data chunk received by udp proxy using currently used load balancing policy.

The UDP proxy listener filter also can operate as a transparent proxy if the use_original_src_ip field is set to true. But please keep in mind that it does not forward the port to upstreams. It forwards only the IP address to upstreams.

Load balancing and unhealthy host handling

Envoy will fully utilize the configured load balancer for the configured upstream cluster when load balancing UDP datagrams. By default, when a new session is created, Envoy will associate the session with an upstream host selected using the configured load balancer. All future datagrams that belong to the session will be routed to the same upstream host. However, if use_per_packet_load_balancing field is set to true, Envoy selects another upstream host on next datagram using the configured load balancer and creates a new session if such does not exist. So in case of several upstream hosts available for the load balancer each data chunk is forwarded to a different host.

When an upstream host becomes unhealthy (due to active health checking), Envoy will attempt to create a new session to a healthy host when the next datagram is received.

Circuit breaking

The number of sessions that can be created per upstream cluster is limited by the cluster’s maximum connection circuit breaker. By default this is 1024.

Routing

The filter can route different datagrams to different upstream clusters with their source addresses. The matching API can be used with UDP routing, by specifying a matcher, a UDP network input as the matching rule and Route as the resulting action.

The following matcher configuration will lead Envoy to route UDP datagrams according to their source IPs, ignore datagrams other than those with a source IP of 127.0.0.1, and then filter the remaining datagrams to different clusters according to their source ports.

14        matcher:
15          # The outer matcher list matches source IP.
16          matcher_list:
17            matchers:
18            - predicate:
19                single_predicate:
20                  input:
21                    name: envoy.matching.inputs.source_ip
22                    typed_config:
23                      '@type': type.googleapis.com/envoy.extensions.matching.common_inputs.network.v3.SourceIPInput
24                  value_match:
25                    exact: 127.0.0.1
26              on_match:
27                matcher:
28                  # The inner matcher tree matches source port.
29                  matcher_tree:
30                    input:
31                      name: envoy.matching.inputs.source_port
32                      typed_config:
33                        '@type': type.googleapis.com/envoy.extensions.matching.common_inputs.network.v3.SourcePortInput
34                    exact_match_map:
35                      map:
36                        "80":
37                          action:
38                            name: route
39                            typed_config:
40                              '@type': type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.Route
41                              cluster: udp_service
42                        "443":
43                          action:
44                            name: route
45                            typed_config:
46                              '@type': type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.Route
47                              cluster: udp_service2
48                  on_no_match:
49                    action:
50                      name: route
51                      typed_config:
52                        '@type': type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.Route
53                        cluster: udp_service3

Session filters

The UDP proxy is able to apply session filters. These kinds of filters run seprately on each upstream UDP session and support a more granular API that allows running operations only at the start of an upstream UDP session, when a UDP datagram is received from the downstream and when a UDP datagram is received from the upstream, similar to network filters.

Note

When using session filters, choosing the upstream host only happens after completing the onNewSession() iteration for all the filters in the chain. This allows choosing the host based on decisions made in one of the session filters, and prevents the creation of upstream sockets in cases where one of the filters stopped the filter chain. Additionally, since per packet load balancing require choosing the upstream host for each received datagram, session filters can’t be used when this option is enabled.

Envoy has the following builtin UDP session filters.

UDP Tunneling over HTTP

The UDP proxy filter can be used to tunnel raw UDP over HTTP requests. Refer to HTTP upgrades for more information.

UDP tunneling configuration can be used by setting tunneling_config

Note

Since per packet load balancing require choosing the upstream host for each received datagram, tunneling can’t be used when this option is enabled.

Example configuration

The following example configuration will cause Envoy to listen on UDP port 1234 and proxy to a UDP server listening on port 1235, allowing 9000 byte packets in both directions (i.e., either jumbo frames or fragmented IP packets).

 1admin:
 2  address:
 3    socket_address:
 4      protocol: TCP
 5      address: 127.0.0.1
 6      port_value: 9901
 7static_resources:
 8  listeners:
 9  - name: listener_0
10    address:
11      socket_address:
12        protocol: UDP
13        address: 127.0.0.1
14        port_value: 1234
15    udp_listener_config:
16      downstream_socket_config:
17        max_rx_datagram_size: 9000
18    listener_filters:
19    - name: envoy.filters.udp_listener.udp_proxy
20      typed_config:
21        '@type': type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.UdpProxyConfig
22        stat_prefix: service
23        matcher:
24          on_no_match:
25            action:
26              name: route
27              typed_config:
28                '@type': type.googleapis.com/envoy.extensions.filters.udp.udp_proxy.v3.Route
29                cluster: service_udp
30        upstream_socket_config:
31          max_rx_datagram_size: 9000
32  clusters:
33  - name: service_udp
34    type: STATIC
35    lb_policy: ROUND_ROBIN
36    load_assignment:
37      cluster_name: service_udp
38      endpoints:
39      - lb_endpoints:
40        - endpoint:
41            address:
42              socket_address:
43                address: 127.0.0.1
44                port_value: 1235

Statistics

The UDP proxy filter emits both its own downstream statistics as well as many of the cluster upstream statistics where applicable. The downstream statistics are rooted at udp.<stat_prefix>. with the following statistics:

Name

Type

Description

downstream_sess_no_route

Counter

Number of datagrams not routed due to no cluster

downstream_sess_rx_bytes

Counter

Number of bytes received

downstream_sess_rx_datagrams

Counter

Number of datagrams received

downstream_sess_rx_errors

Counter

Number of datagram receive errors

downstream_sess_total

Counter

Number sessions created in total

downstream_sess_tx_bytes

Counter

Number of bytes transmitted

downstream_sess_tx_datagrams

Counter

Number of datagrams transmitted

downstream_sess_tx_errors

Counter

Number of datagram transmission errors

idle_timeout

Counter

Number of sessions destroyed due to idle timeout

downstream_sess_active

Gauge

Number of sessions currently active

The following standard upstream cluster stats are used by the UDP proxy:

Name

Type

Description

upstream_cx_none_healthy

Counter

Number of datagrams dropped due to no healthy hosts

upstream_cx_overflow

Counter

Number of datagrams dropped due to hitting the session circuit breaker

upstream_cx_rx_bytes_total

Counter

Number of bytes received

upstream_cx_tx_bytes_total

Counter

Number of bytes transmitted

The UDP proxy filter also emits custom upstream cluster stats prefixed with cluster.<cluster_name>.udp.:

Name

Type

Description

sess_rx_datagrams

Counter

Number of datagrams received

sess_rx_datagrams_dropped

Counter

Number of datagrams dropped due to kernel overflow or truncation

sess_rx_errors

Counter

Number of datagram receive errors

sess_tx_datagrams

Counter

Number of datagrams transmitted

sess_tx_errors

Counter

Number of datagrams transmitted

sess_tunnel_success

Counter

Number of successfully established UDP tunnels

sess_tunnel_failure

Counter

Number of UDP tunnels failed to establish

sess_tunnel_buffer_overflow

Counter

Number of datagrams dropped due to full tunnel buffer