Client-Side Weighted Round Robin Load Balancing Policy (proto)

This extension has the qualified name envoy.load_balancing_policies.client_side_weighted_round_robin

Note

This extension is functional but has not had substantial production burn time, use only with this caveat.

This extension has an unknown security posture and should only be used in deployments where both the downstream and upstream are trusted.

Tip

This extension extends and can be used with the following extension category:

envoy.load_balancing_policies

This extension must be configured with one of the following type URLs:

type.googleapis.com/envoy.extensions.load_balancing_policies.client_side_weighted_round_robin.v3.ClientSideWeightedRoundRobin

extensions.load_balancing_policies.client_side_weighted_round_robin.v3.ClientSideWeightedRoundRobin

[extensions.load_balancing_policies.client_side_weighted_round_robin.v3.ClientSideWeightedRoundRobin proto]

Configuration for the client_side_weighted_round_robin LB policy.

This policy differs from the built-in ROUND_ROBIN policy in terms of how the endpoint weights are determined. In the ROUND_ROBIN policy, the endpoint weights are sent by the control plane via EDS. However, in this policy, the endpoint weights are instead determined via qps (queries per second), eps (errors per second), and utilization metrics sent by the endpoint using the Open Request Cost Aggregation (ORCA) protocol. All queries count toward qps, regardless of result. Only failed queries count toward eps. A config parameter error_utilization_penalty controls the penalty to adjust endpoint weights using eps and qps. The weight of a given endpoint is computed as: qps / (utilization + eps/qps * error_utilization_penalty).

For a load report to update an endpoint’s weight, it must set rps_fractional (used as qps) greater than 0, and the final utilization (resolved utilization plus any error penalty) must be greater than 0. Resolved utilization (see metric_names_for_computing_utilization) is used as the baseline. Reports that fail to do so are ignored, and an endpoint with no valid weight is assigned the median weight of the endpoints that have one.

Note that Envoy will forward the ORCA response headers/trailers from the upstream cluster to the downstream client. This means that if the downstream client is also configured to use client_side_weighted_round_robin it will load balance against Envoy based on upstream weights. This can happen when Envoy is used as a reverse proxy. To avoid this issue you can configure the header_mutation filter to remove the ORCA payload from the response headers/trailers.

See the load balancing architecture overview for more information.

{
  "enable_oob_load_report": {...},
  "oob_reporting_period": {...},
  "blackout_period": {...},
  "weight_expiration_period": {...},
  "weight_update_period": {...},
  "error_utilization_penalty": {...},
  "metric_names_for_computing_utilization": [],
  "slow_start_config": {...},
  "oob_reporting_config": {...}
}

enable_oob_load_report: (BoolValue) Whether to enable out-of-band utilization reporting collection from the endpoints. By default, per-request utilization reporting is used.

oob_reporting_period: (Duration) Load reporting interval to request from the server. Note that the server may not provide reports as frequently as the client requests. Used only when enable_oob_load_report is true. Default is 10 seconds.

blackout_period: (Duration) A given endpoint must report load metrics continuously for at least this long before the endpoint weight will be used. This avoids churn when the set of endpoint addresses changes. Takes effect both immediately after we establish a connection to an endpoint and after weight_expiration_period has caused us to stop using the most recent load metrics. Default is 10 seconds.

weight_expiration_period: (Duration) If a given endpoint has not reported load metrics in this long, then we stop using the reported weight. This ensures that we do not continue to use very stale weights. Once we stop using a stale value, if we later start seeing fresh reports again, the blackout_period applies. Defaults to 3 minutes.

weight_update_period: (Duration) How often endpoint weights are recalculated. Values less than 100ms are capped at 100ms. Default is 1 second.

error_utilization_penalty: (FloatValue) The multiplier used to adjust endpoint weights with the error rate calculated as eps/qps. Configuration is rejected if this value is negative. Default is 1.0.

metric_names_for_computing_utilization: (repeated string) Specifies the metrics used to compute the endpoint utilization from which weight is derived. For map fields in the ORCA proto, the string will be of the form <map_field_name>.<map_key>. For example, the string named_metrics.foo will mean to look for the key foo in the ORCA named_metrics field. Utilization is the max of the values of the metrics specified here, when that max is greater than 0. Otherwise application_utilization is used if greater than 0, with cpu_utilization as the final fallback. Disabling the runtime flag envoy.reloadable_features.orca_weight_manager_use_named_metrics_first restores the legacy order, preferring application_utilization over these metrics.

slow_start_config: (extensions.load_balancing_policies.common.v3.SlowStartConfig) Configuration for slow start mode. If this configuration is not set, slow start will not be not enabled.

oob_reporting_config: (extensions.load_balancing_policies.common.v3.OrcaOobReportingConfig) Optional overrides for the OOB reporting connection (alternative port, :authority, transport socket selection). Honored only when enable_oob_load_report is true.