Rate limit

  • Global rate limiting architecture overview

  • This filter should be configured with the type URL type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit.

  • v3 API reference

The HTTP rate limit filter will call the rate limit service when the request’s route or virtual host has one or more rate limit configurations that match the filter stage setting. The route can optionally include the virtual host rate limit configurations. More than one configuration can apply to a request. Each configuration results in a descriptor being sent to the rate limit service.

If the rate limit service is called, and the response for any of the descriptors is over limit, a 429 response is returned (the response is configurable via rate_limited_status). The rate limit filter also sets the x-envoy-ratelimited header, unless disable_x_envoy_ratelimited_header is set to true.

If there is an error in calling rate limit service or rate limit service returns an error and failure_mode_deny is set to true, a 500 response is returned.

Composing Actions

Each rate limit action on the route or virtual host populates a descriptor entry. A vector of descriptor entries compose a descriptor. To create more complex rate limit descriptors, actions can be composed in any order. The descriptor will be populated in the order the actions are specified in the configuration.

Example 1

For example, to generate the following descriptor:

("generic_key", "some_value")
("source_cluster", "from_cluster")

The configuration would be:

actions:
    - {source_cluster: {}}
    - {generic_key: {descriptor_value: some_value}}

Example 2

If an action doesn’t append a descriptor entry, no descriptor is generated for the configuration.

For the following configuration:

actions:
    - {source_cluster: {}}
    - {remote_address: {}}
    - {generic_key: {descriptor_value: some_value}}

If a request did not set x-forwarded-for, no descriptor is generated.

If a request sets x-forwarded-for, the the following descriptor is generated:

("generic_key", "some_value")
("remote_address", "<trusted address from x-forwarded-for>")
("source_cluster", "from_cluster")

Rate Limit Override

A rate limit action can optionally contain a limit override. The limit value will be appended to the descriptor produced by the action and sent to the ratelimit service, overriding the static service configuration.

The override can be configured to be taken from the Dynamic Metadata under a specified key. If the value is misconfigured or key does not exist, the override configuration is ignored.

Example 3

The following configuration

actions:
    - {generic_key: {descriptor_value: some_value}}
limit:
   metadata_key:
       key: test.filter.key
       path:
           - key: test

Will lookup the value of the dynamic metadata. The value must be a structure with integer field “requests_per_unit” and a string field “unit” which is parseable to RateLimitUnit enum. For example, with the following dynamic metadata the rate limit override of 42 requests per hour will be appended to the rate limit descriptor.

test.filter.key:
    test:
        requests_per_unit: 42
        unit: HOUR

Descriptor extensions

Rate limit descriptors are extensible with custom descriptors. For example, computed descriptors extension allows using any of the request attributes as a descriptor value:

actions:
    - extension:
          name: custom
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.rate_limit_descriptors.expr.v3.Descriptor
            descriptor_key: my_descriptor_name
            text: request.method

HTTP matching input functions are supported as descriptor producers:

actions:
    - extension:
          name: custom
          typed_config:
              "@type": type.googleapis.com/envoy.type.matcher.v3.HttpRequestHeaderMatchInput
              header_name: x-header-name

The above example produces an entry with the key custom and the value of the request header x-header-name. If the header is absent, then the descriptor entry is not produced, and no descriptor is generated. If the header value is present but is an empty string, then the descriptor is generated but no entry is added.

Statistics

The rate limit filter outputs statistics in the cluster.<route target cluster>.ratelimit.<optional stat prefix>. namespace. 429 responses or the configured rate_limited_status are emitted to the normal cluster dynamic HTTP statistics.

Name

Type

Description

ok

Counter

Total under limit responses from the rate limit service

error

Counter

Total errors contacting the rate limit service

over_limit

Counter

total over limit responses from the rate limit service

failure_mode_allowed

Counter

Total requests that were error(s) but were allowed through because of failure_mode_deny set to false.

Dynamic Metadata

The ratelimit filter emits dynamic metadata as an opaque google.protobuf.Struct only when the gRPC ratelimit service returns a RateLimitResponse with a filled dynamic_metadata field.

Runtime

The HTTP rate limit filter supports the following runtime settings:

ratelimit.http_filter_enabled

% of requests that will call the rate limit service. Defaults to 100.

ratelimit.http_filter_enforcing

% of requests that that will have the rate limit service decision enforced. Defaults to 100. This can be used to test what would happen before fully enforcing the outcome.

ratelimit.<route_key>.http_filter_enabled

% of requests that will call the rate limit service for a given route_key specified in the rate limit configuration. Defaults to 100.