Override Host Load Balancing Policy (proto)

This extension has the qualified name envoy.load_balancing_policies.override_host

Note

This extension is functional but has not had substantial production burn time, use only with this caveat.

This extension has an unknown security posture and should only be used in deployments where both the downstream and upstream are trusted.

Tip

This extension extends and can be used with the following extension category:

envoy.load_balancing_policies

This extension must be configured with one of the following type URLs:

type.googleapis.com/envoy.extensions.load_balancing_policies.override_host.v3.OverrideHost

extensions.load_balancing_policies.override_host.v3.OverrideHost

[extensions.load_balancing_policies.override_host.v3.OverrideHost proto]

Configuration for the Override Host Load Balancing policy.

This policy allows endpoint picking to be implemented in an external to Envoy component. For example an ext_proc RPC to a service that implements k8s proposal for AI gateway inferences extensions https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/docs/proposals/004-endpoint-picker-protocol can provide hosts for serving a request using Override Host load balancing policy.

This policy uses selected endpoints from either request headers or request metadata.

The primary host override source must specify a single IP:port value in either the header or the metadata. For example 10.0.0.5:8080 or [2600:4040:5204::1574:24ae]:80. The IPv6 address is enclosed in square brackets.

For example, to support k8s gateway inference extensions, which uses the x-gateway-destination-endpoint header or metadata value under the “envoy.lb” key for the primary endpoint, the Override Host load balancing policy should be configured in the following way:

override_host_sources:
  - header: "x-gateway-destination-endpoint"
  - metadata:
      key: "envoy.lb"
      path:
      - key: "x-gateway-destination-endpoint"
  - header: "x-gateway-fallback-endpoints"

If neither header nor metadata is present, then the specified fallback load balancing policy is used. This allows load balancing to degrade to a a built in policy (i.e. Round Robin) in case external endpoint picker fails.

See the load balancing architecture overview for more information.

{
  "override_host_sources": [],
  "fallback_policy": {...}
}

override_host_sources: (repeated extensions.load_balancing_policies.override_host.v3.OverrideHost.OverrideHostSource, REQUIRED) A list of sources to get host addresses from. The host sources are searched in the order specified. The request is forwarded to the first address and subsequent addresses are used for request retries or hedging. Note that if an overridden host address is not present in the current endpoint set, it is skipped and the next found address is used. If there are not enough overridden addresses to satisfy all retry attempts the fallback load balancing policy is used to pick a host.

fallback_policy: (config.cluster.v3.LoadBalancingPolicy, REQUIRED) The child LB policy to use in case neither header nor metadata with selected hosts is present.

extensions.load_balancing_policies.override_host.v3.OverrideHost.OverrideHostSource

[extensions.load_balancing_policies.override_host.v3.OverrideHost.OverrideHostSource proto]

{
  "header": ...,
  "metadata": {...}
}

header: (string) The header to get the override host addresses. Only one of the header or metadata field must be set.

metadata: (type.metadata.v3.MetadataKey) The metadata key to get the override host addresses from the request dynamic metadata. If set this field then it will take precedence over the header field. Only one of the header or metadata field must be set.