The network filter could be used to add multiple protocols support to Envoy. The community has implemented lots of this kind of network filter, such as Dubbo proxy, Thrift proxy, etc. Developers may also implement their own protocol proxying by creating a new network filter.
Adding a new network filter to support a new protocol is not easy. Developers need to implement the codec to parse the binary, to implement the stream management to handle request/response, to implement the connection management to handle connection lifecycle, etc. There are many common features like route, L7 filter chain, tracing, metrics, logging, etc. that need to be implemented again and again in each new network filter.
Many RPC protocols has a similar request/response based architecture. The request could be routed to different upstreams based on the request properties. For these similar protocols, their network filters also are similar and could be abstracted to following modules:
Codec: parse the binary to request/response object.
Stream management: handle the request/response stream.
Route: route the request to different upstream based on the request properties.
L7 filter chain: filter the request/response.
Observability: tracing, metrics, logging, etc.
With the exception of the codec, all other modules are common and could be shared by different protocols. This is the motivation of generic proxy.
Generic proxy is a network filter that could be used to implement new protocol proxying. An extension point is provided to let the users configure specific codec. The developers only need to implement the codec to parse the binary and let users configure the codec in the generic proxy. The generic proxy will handle the rest of work.
Abstraction of the request/response is the core of generic proxy. Different L7 protocols may have a different request/response data structure. In the generic proxy, the codec is extended and could be configured by users, but other modules are common and can be shared by different protocols. These different request/response data structures of different protocols need to be abstracted and managed in a common abstraction.
Request class is defined to represent the request. The
Request provides some virtual methods to get/set the request properties.
Different protocols can extend the
Request and implement the virtual methods to get/set the request properties in their own data structure.
An abstract class
Response is defined to represent the response. It is similar to
Based on the abstraction of the request/response, the generic proxy can handle the request/response stream, route the request to different upstreams, filter the request/response, etc. without needing to know the L7 application and specific data structure of the request/response.
If the developers want to implement a new protocol proxying, they only need to implement the codec to parse the binary data to specific request/response
and ensure they implement
Response. It is much easier than implement a new network filter.
Extendable matcher and route
Generic matcher API is used to construct the route table of the generic proxy. The developers could extend input and matcher to support new matching logic.
By default, the generic proxy supports following input and matcher:
host: match the host of the request. The host should represents set of instances that can be used for load balancing. It could be DNS name or VIP of the target service.
path: match the path of the request. The path should represents RPC service name that used to represents set of method or functionality provided by target service.
method: match the method of the request.
property: match the property of the request. The property could be any property of the request that indexed by string key.
request input and request matcher: match the whole request by combining host, path, method and properties in AND semantics. This is used to match multiple fields of the downstream request and avoid complex combinations of host, path, method and properties in the generic matching tree.
Virtual methods are used to get the request properties in
method(), etc. The developers need to
implement these virtual methods and determine what these fields are in their own protocol. It is possible that a protocol does not have some of
these fields. For example, the protocol does not have
host field. In this case, the developers could return empty string in the virtual method
to indicate the field does not exist in the protocol. The only drawback is that the generic proxy could not match the request by this field.
21 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.codecs.dubbo.v3.DubboCodecConfig 22 route_config: 23 name: route_config 24 virtual_hosts: 25 - name: route_config_default_virtual_host 26 hosts: 27 - "org.apache.dubbo.UserProvider" 28 routes: 29 matcher_list: 30 matchers: 31 - predicate: 32 single_predicate: 33 input: 34 name: request 35 typed_config: 36 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.matcher.v3.RequestMatchInput 37 custom_match: 38 name: request 39 typed_config: 40 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.matcher.v3.RequestMatcher 41 host: 42 exact: "org.apache.dubbo.UserProvider" 43 method: 44 exact: "getUser" 45 properties: 46 - name: "id" 47 string_match: 48 exact: "1" 49 on_match: 50 action: 51 name: route 52 typed_config: 53 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.action.v3.RouteAction
Asynchronous codec API
The generic proxy provides an extension point to let the developers implement a codec specific to their own protocol. The codec API is designed to be asynchronous to avoid blocking the worker thread. And the asynchronous codec API makes it possible to accelerate the codec by offloading the parsing work to specific hardware.
Different protocols may have different connection lifecycles or connection management. The generic proxy provides additional options to let codec developers configure the connection lifecycle and connection management.
For example, developers can configure whether the upstream connection is bound to the downstream connection or not. If the upstream connection is bound to the downstream connection, the upstream connection will have same lifetime as the downstream connection. The bound upstream connection will be used only by requests that come from the related downstream connection. This is useful for the protocols that need to keep the connection state.
Developers can also operate the downstream connection and upstream connection in the codec directly. This gives developers more control over the connection.
Example codec implementation
The community has implemented a dubbo codec based on generic proxy. The dubbo codec is a good example showing how to implement a new codec for new protocol because of its moderate complexity.
You could find the dubbo codec implementation in contrib/generic_proxy/filters/network/source/codecs/dubbo directory. You can also configure the dubbo codec in the generic proxy with the following configuration:
1static_resources: 2 listeners: 3 - name: main 4 address: 5 socket_address: 6 address: 0.0.0.0 7 port_value: 9090 8 filter_chains: 9 - filters: 10 - name: generic_proxy 11 typed_config: 12 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.v3.GenericProxy 13 stat_prefix: stats_prefix 14 filters: 15 - name: envoy.filters.generic.router 16 typed_config: 17 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.router.v3.Router 18 codec_config: 19 name: dubbo 20 typed_config: 21 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.codecs.dubbo.v3.DubboCodecConfig 22 route_config: 23 name: route_config 24 virtual_hosts: 25 - name: route_config_default_virtual_host 26 hosts: 27 - "org.apache.dubbo.UserProvider" 28 routes: 29 matcher_list: 30 matchers: 31 - predicate: 32 single_predicate: 33 input: 34 name: request 35 typed_config: 36 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.matcher.v3.RequestMatchInput 37 custom_match: 38 name: request 39 typed_config: 40 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.matcher.v3.RequestMatcher 41 host: 42 exact: "org.apache.dubbo.UserProvider" 43 method: 44 exact: "getUser" 45 properties: 46 - name: "id" 47 string_match: 48 exact: "1" 49 on_match: 50 action: 51 name: route 52 typed_config: 53 "@type": type.googleapis.com/envoy.extensions.filters.network.generic_proxy.action.v3.RouteAction 54 cluster: dubbo 55 clusters: 56 - name: dubbo 57 connect_timeout: 5s 58 type: STRICT_DNS 59 load_assignment: 60 cluster_name: dubbo 61 endpoints: 62 - lb_endpoints: 63 - endpoint: 64 address: 65 socket_address: 66 address: localhost 67 port_value: 8080