1.40.0 (Pending)
Incompatible behavior changes
Changes that are expected to cause an incompatibility if applicable; deployment changes are likely required
build: Bumped the hermetic LLVM/Clang toolchain from 18 to 22. This upgrades the default compiler used by
--config=clangand may surface new warnings or diagnostics in downstream builds that pin to the Envoy toolchain.ssl: Removed AWS-LC as a selectable SSL library, along with the
--config=aws-lc-fipsbuild configuration. AWS-LC was previously the only way to build for the ppc64le architecture; ppc64le builds should now use--config=opensslinstead.
Minor behavior changes
Changes that may cause incompatibilities for some users, but should not for most
dynamic_forward_proxy: DNS cache statistics (
dns_cache.*) are now always created under the server-wide stats scope instead of a caller-derived scope captured when the manager singleton was first instantiated. As a result they are now matched against the global stats matcher rather than a per-listener stats matcher. Statistic names are unchanged, so this only affects configurations where the listener that first created a DNS cache declared a per-listener stats matcher.ext_proc: The external processing filter now logs the target URI as the destination when using the
google_grpcservice. Previously, only the cluster name of theenvoy_grpcservice was logged.happy_eyeballs: The happy eyeballs sorting of a multi-address host’s address list now happens once when the address list is created or refreshed, instead of on every upstream connection attempt. The order in which connection attempts are made is unchanged.
redis_proxy: The Redis proxy codec is stricter about malformed RESP wire input that was previously accepted silently: negative aggregate or bulk length headers other than the spec’s
*-1/$-1null forms, integer lines carrying no digits, integers outside the signed 64-bit range, and messages exceeding new nesting-depth, cumulative-element, inline-command-element and scalar-token limits are now treated as protocol errors that close the connection. A single bulk string, blob error or verbatim string payload is additionally capped at 512 MiB, matching Redis’s defaultproto-max-bulk-len; deployments whose backends raiseproto-max-bulk-lenbeyond that default are affected by this cap. Locally generated error replies also have ASCII control bytes replaced with spaces so attacker-influenced text cannot inject RESP framing. The remaining changes are only visible to peers sending non-conforming or abusive wire data.server: Fixed container-aware CPU limit detection (#45410) not being enabled by default. The minimum of the cgroup CPU limit, CPU affinity, and hardware thread count, added in #40997 and documented as the default in the v1.37.0 release notes, was only applied when
--cpuset-threadswas set. It is now applied whenever--concurrencyis not set, so worker threads are sized to the cgroup CPU limit in containerized deployments without requiring--cpuset-threads. Detection can still be disabled by settingENVOY_CGROUP_CPU_DETECTIONtofalse.stats: Added the runtime guard
envoy.reloadable_features.enable_stats_explicit_tags(defaultfalse). When set totrueand the stats configuration carries no custom tags (empty stats_tags and use_all_default_tags left at its default oftrue), the stats store uses the tags supplied by the calling code (the explicit-tags logic) and propagates scope-level tags onto every stat, instead of re-parsing the flat stat name. The guard is evaluated once at startup. There is no visible change to users while the guard remainsfalse.upstream: The runtime guard
envoy.reloadable_features.coalesce_lb_rebuilds_on_batch_updatenow defaults totrue. A thread-aware load balancer (for exampleRING_HASHorMAGLEV) rebuilds its factory state once at the end of a batch host update, from the single end-of-cycle member-update callback, instead of once per priority from the per-priority update callback. The rebuild still lands before the cluster manager posts the update to the worker threads, so this only removes the redundant per-priority rebuilds of a batch. This can be reverted by settingenvoy.reloadable_features.coalesce_lb_rebuilds_on_batch_updatetofalse.watchdog: Configuring the envoy.watchdog.backtrace_action now causes Envoy to install a process-wide
SIGUSR2signal handler and to sendSIGUSR2to stuck threads in order to capture their backtraces. Deployments that rely onSIGUSR2for other purposes should avoid enabling this action.
Bug fixes
Changes expected to improve the state of the world and are unlikely to have negative effects
access_log: Fixed a bug where omit_empty_values had no effect for
json_format. Because the JSON formatter pre-serializes the template when loading the configuration, keys whose command operators evaluated to null were still emitted (for example{"key":null}instead of{}). Whenomit_empty_valuesis set, the JSON formatter now omits keys with null values, removes nested objects that become empty, and preserves empty arrays, matching the documented behavior. This behavioral change can be reverted by setting the runtime guardenvoy.reloadable_features.json_formatter_omit_empty_valuestofalse.credential_injector: Fixed a bug where a credential loaded from a file-based generic secret was injected into the request header verbatim, including any trailing newline commonly present in secret files. Since HTTP header values cannot contain CR/LF, this produced an invalid header and the request failed. Trailing CR/LF characters are now stripped from the credential before injection, and a credential consisting only of CR/LF characters is treated as missing.
dynamic_forward_proxy: Fixed a use-after-free crash in the DNS cache manager: the server-wide
DnsCacheManagersingleton no longer retains the stats scope of the listener or filter chain that first created a DNS cache, which could be freed before a later cache miss for a new cache name dereferenced it.ext_proc: Fixed multiple lifetime bugs in the external processing (
ext_proc) filter and the underlying gRPC async client that could lead to use-after-free or double delivery of callbacks. The gRPC async client now holds an optional reference to its stream callbacks and drops it once the stream is cleaned up or the owner detaches viawaitForRemoteCloseAndDelete(), so a stream that outlives its callbacks (for example while awaiting remote close) no longer invokes callbacks on freed memory. Re-entrant resets during stream initialization are guarded so remote close is not notified (and the tracing span not finished) twice when the cluster is missing or stream creation fails synchronously, and half-close/cleanup no longer dereference a stream that was never established. Theext_procThreadLocalStreamManagerandProcessorStreamImplnow close any still-open streams on destruction to avoid dangling references into the underlying gRPC stream.grpc_http1_reverse_bridge: Fixed a crash (SEGFAULT) in the
grpc_http1_reverse_bridgefilter whenwithhold_grpc_framesis enabled withoutresponse_size_headerand the upstream response body exceeds the downstream HTTP/2 stream flow control window. The filter now uses the upstreamContent-Lengthheader to stream the response incrementally instead of buffering and releasing it all at once.http: Fixed a bug where malformed CONNECT request lines without an authority could cause the legacy HTTP/1 parser to encode a
400 Bad Requestresponse using HTTP/1.0. Envoy now rejects these requests without downgrading the response protocol.http: Fixed a bug where response metadata added by HTTP encoder filters could be dropped when a later encoder filter sent a direct local reply before final response headers were encoded to the codec. Saved response metadata is now flushed before the local reply ends the stream. This behavior can be temporarily reverted by setting the runtime guard
envoy.reloadable_features.direct_local_reply_flush_saved_response_metadatatofalse.http2: Fixed an integer overflow in HTTP/2 codec stream flow control accounting where
unconsumed_bytes_wrapped around when reads were disabled on a stream receiving over 4 GiB of data.listener: Fixed a crash at startup when a UDP or QUIC listener was configured with
bind_to_port: false. This combination was never functional and is now rejected at configuration load with a validation error.mcp: Fixed a memory usage issue in the MCP JSON-RPC parser by optimizing node allocation for unneeded fields.
mcp_json_rest_bridge: Fixed a path-traversal issue in the
mcp_json_rest_bridgeHTTP filter where a path-template variable’s value (taken from attacker-controlled tool-call arguments) was installed verbatim into the upstream request:path, so a value such as../../admin/secretsproduced raw path traversal. Traversal segments (./..) are now rejected for every template variable, and a “simple” variable (for example{id}) additionally has/percent-encoded to confine it to a single path segment. Variables with an explicit pattern such as{name=projects/*}may still legitimately span multiple segments.mcp_json_rest_bridge: mcp_json_rest_bridge: Fixed a bug where headers-only upstream responses (e.g., HTTP 204 No Content) were passed through to MCP clients without a JSON-RPC response body, causing MCP SDK timeouts or exceptions. The filter now synthesizes a valid JSON-RPC response: an empty
ToolResultfortools/callrequests and a server error fortools/listrequests.open_telemetry: Fixed the OpenTelemetry access loggers (both the gRPC and HTTP variants) ignoring configured formatters when building custom_tags. Previously a custom tag whose value used a formatter extension command failed with
Not supported field in StreamInfo, even though the same command worked inbodyandattributes. The configured command parsers are now passed through to custom-tag creation.redis_proxy: Fixed a use-after-free in the Redis cluster
CLUSTER SLOTSdiscovery. A cluster refresh (periodic resolve timer or DNS update) that arrived afterCLUSTER SLOTScompleted but while the zone-discoveryINFOrequests it triggered were still in flight could start a second discovery, overwrite the in-flight callbacks and free memory still referenced by the outstanding requests.reverse_tunnel: Fixed a bug in the reverse tunnel downstream socket interface (
envoy.bootstrap.reverse_tunnel.downstream_socket_interface) where handshakeadditional_headersvalues that use aThreadLocal-backed substitution formatter (such as%FILE_CONTENT%, or secret/SDS-backed formatters) resolved to an empty string on the worker thread that assembles the handshake request. The handshake formatters were built in the bootstrap extension constructor, which runs before the worker threads register with theThreadLocalsystem, so the formatter providers’ thread-local slots were never populated on the workers. The formatters are now built inonServerInitialized(), after the workers are registered, so their values propagate to every worker thread.reverse_tunnel: Fixed a bug in the reverse tunnel downstream socket interface (
envoy.bootstrap.reverse_tunnel.downstream_socket_interface) where handshakeadditional_headersvalues that use a substitution formatter (such as%FILE_CONTENT%, or secret/SDS-backed formatters) were sent as the raw, unsubstituted template on every reverse connection. The reverse connection listen socket snapshots the handshake formatters when it is created, which can happen beforeonServerInitialized()builds them; that null snapshot is then reused for every re-dial, so the handshake fell back to emitting the literaladditional_headersvalue. The handshake headers are now resolved from the live bootstrap extension when the request is assembled, so post-initialization dials substitute the value correctly.thrift_proxy: Fixed a 32-bit integer overflow in the
thrift_proxylax (non-strict) binary protocol decoder. A message name length of 0xFFFFFFF7 or greater wrapped the insufficient-data check inreadMessageBeginand raised a spurious decode error that closed the downstream connection. The check is now performed in 64-bit arithmetic and the decoder waits for more data instead, matching the strict binary protocol.tls: Fixed a memory leak in the OpenSSL compatibility layer where
SSL_get0_peer_certificates()calledSSL_get_peer_certificate()without freeing the returned reference. Each call leaked oneX509refcount, preventing the certificate and its sub-allocations from being freed when the connection closed, causing unbounded memory growth in certain deployments.tls: Fixed upstream TLS client session caching so sessions are scoped by the effective SNI used for the connection. This prevents a session learned for one upstream SNI from being offered on a connection using a different SNI. The existing
max_session_keyssetting continues to limit the total number of cached sessions. This behavior can be temporarily reverted by setting runtime guardenvoy.reloadable_features.scope_upstream_tls_session_cache_by_snitofalse.upstream: Fixed a bug where
upstream_bind_configwith port0could cause ephemeral port exhaustion by reserving an ephemeral port duringbind(). Envoy now automatically enablesIP_BIND_ADDRESS_NO_PORTto defer port allocation untilconnect(). This change can be temporarily reverted by setting runtime guardenvoy.reloadable_features.upstream_bind_config_fix_port_exhaustiontofalse.upstream: Fixed a race condition affecting thread-aware load balancers (for example
RING_HASHandMAGLEV) where, after a transient health-check failure followed by an immediate recovery, a worker thread could snapshot a stale load balancer factory and leave the recovered host absent from the ring/table until the next membership change. The thread-aware load balancer now rebuilds its factory state before the cluster manager posts the corresponding host update to the worker threads, so a worker can no longer snapshot a stale factory. Whenenvoy.reloadable_features.enable_batch_aware_updateis enabled (the default), the cluster manager accumulates per-priority host updates and posts them to the worker threads from the single end-of-cycle member-update callback (once for a whole batch host update, once after each individual update), instead of posting once per priority; mergeable health-check/weight/metadata updates still flow through the update merge window. The accumulated update is applied to each worker thread’s priority set as a single batch so the worker-local load balancer rebuilds once for the whole update instead of once per priority. This can be reverted by settingenvoy.reloadable_features.enable_batch_aware_updatetofalse. The thread-aware load balancer rebuilds its factory from the priority-update callback; whenenvoy.reloadable_features.coalesce_lb_rebuilds_on_batch_updateis also enabled it instead defers the rebuild to the single end-of-cycle member-update callback, coalescing the per-priority rebuilds of a batch into one (which still lands before the batched post).
Removed config or runtime
Normally occurs at the end of the deprecation period
generic_proxy: Removed the runtime guard
envoy.reloadable_features.generic_proxy_codec_buffer_limitand the legacy code path it guarded. The generic proxy Dubbo, HTTP/1 and Kafka codecs now always fail decoding when the buffered data exceeds the connection buffer limit.http2: Removed the runtime guard
envoy.reloadable_features.safe_http2_optionsand the legacy code path it guarded. HTTP/2 connections now always fall back to the safe defaults (max concurrent streams of 1024, 16 MiB initial stream window and 24 MiB initial connection window) when the corresponding options are unset, and the unused legacy default constants are removed.oauth2: Removed the runtime guard
envoy.reloadable_features.oauth2_cleanup_cookiesand the legacy code path it guarded. The OAuth2 filter now always removes the OAuth flow cookies (OauthHMAC,OauthExpires,RefreshToken,OauthNonceandCodeVerifier, including their suffixed names) from a request before it is forwarded upstream, so these cookies are no longer exposed to the backend service.on_demand: Removed the runtime guard
envoy.reloadable_features.on_demand_track_end_streamand the legacy code path it guarded. The on-demand filter now always tracks the downstreamend_streamstate to decide whether a stream with a fully read body can be recreated, instead of rejecting all requests that carry a body.tracing: Removed the runtime guard
envoy.reloadable_features.trace_refresh_after_route_refreshand the legacy code path it guarded. The HTTP connection manager now always refreshes the trace decision and decorator when the route is refreshed, and charges the tracing statistics fromchargeStatsrather than from the old un-refreshed code path.wasm: Removed the runtime guard
envoy.reloadable_features.wasm_use_effective_ctx_for_foreign_functionsand the legacy code path it guarded. Theset_envoy_filter_stateandclear_route_cacheWasm foreign functions now always resolve the effective context (contextOrEffectiveContext) instead of the current context.
New features
access_log: Added the
%LISTENER_NAME%access log command operator, which logs the name of the listener that accepted the downstream connection.access_log: Added the
DS_HS_BEG(downstream TLS handshake begin, i.e. when the ClientHello was received) andDS_HS_END(downstream TLS handshake end) time points to the %COMMON_DURATION% access log formatter. These are populated for both TLS and QUIC downstream connections. Also added the%DOWNSTREAM_CX_RTT%access log formatter returning the last measured round trip time of the downstream connection in milliseconds.admin: Added
invert_filterquery parameter to the/statsand/stats/prometheusadmin endpoints. When set, thefilterregex is inverted so matching stats are excluded from the output (e.g./stats?filter=server&invert_filter).aws_eventstream_parser: Added the aws_eventstream_parser filter. This filter extracts values from AWS EventStream HTTP response bodies (used by AWS Bedrock streaming APIs) and writes them to dynamic metadata for observability, logging, and cost tracking use cases.
compressor: Extended the compressor filter to support usage as an upstream HTTP filter. This can be used to apply request compression for OTLP traffic.
dns_filter: Added case_insensitive to the DNS filter. When set, virtual domain names are matched case-insensitively while the response still echoes the client’s original query-name case. Defaults to
false.dynamic_modules: Added stats sink snapshot getters that expose each metric’s tag-extracted name and its tags (name/value pairs) for counters, gauges, and text readouts, so a dynamic module can reconstruct the dimensional metric names Envoy’s built-in formatters produce. Available through the Rust SDK
MetricSnapshottag accessors.dynamic_modules: Added the
envoy_dynamic_module_callback_listener_filter_set_filter_state_typedandenvoy_dynamic_module_callback_listener_filter_get_filter_state_typedABI callbacks so a dynamic-module listener filter can write and read typed filter state, mirroring the existing bytes setter/getter. Unlike the bytes variant which stores a rawRouter::StringAccessor, the typed setter uses the key’s registeredObjectFactoryto build a properly typed filter state object, so a built-in Envoy filter that reads the key as a typed object can consume it. The Rust SDK exposes these asEnvoyListenerFilter::set_filter_state_typedandEnvoyListenerFilter::get_filter_state_typed.http: Added support for forwarding the issuer of the client certificate in the
x-forwarded-client-cert(XFCC) header via the new issuer field ofSetCurrentClientCertDetails. When enabled, theIssuerkey is added in text format and theissuerfield is added in JSON format. Defaults to disabled.mcp: Added a
NOOPtraffic mode toMcpFilter.mcp_transcoder: Added
per_route_onlyconfig toMcpJsonRestBridge. When set, the filter will take no action unless per-route configuration is available.quic: Added support for memory optimization in QUIC by resetting the internal SSL object after the handshake finishes. This can be enabled by setting the runtime guard
envoy.reloadable_features.quic_enable_reset_ssl_after_handshaketotrue.quic: Upstream QUIC connections now present the client certificate configured in the cluster’s upstream TLS context when the upstream server requests one. Previously configured client certificates were silently not sent over HTTP/3. Client certificates using a private key provider are not supported over QUIC and are now rejected at configuration load time. This behavior change can be reverted by setting the runtime guard
envoy.reloadable_features.quic_upstream_client_certificatestofalse; the guard is evaluated when a cluster’s transport socket is created, so flipping it takes effect on clusters created or updated afterwards.redis_proxy: Added RESP3 protocol support to the Redis proxy via the new protocol_version listener setting (default
RESP2keeps the existing behavior). When set toRESP3, downstream clients negotiate with an explicitHELLO 3handshake — data commands sent beforehand are rejected with-NOPROTOand counted by the newdownstream_rq_noprotocounter — and every new upstream connection performs aHELLO 3handshake (combined withAUTHor AWS IAM credentials when configured, followed byREADONLYwhere applicable) before serving traffic; requests issued during the handshake are held and replayed in order, and negotiation failures are tracked by the new per-clusterupstream_resp3_hello_failurecounter. Independently of the setting, the proxy now answersHELLO,CLIENT SETNAMEandCLIENT SETINFOlocally so that modern Redis clients can complete their connection setup, and the codec understands all RESP3 frame types, down-converting them for RESP2 connections.reverse_tunnel: Added experimental inline JWT authentication for the reverse tunnel handshake via the new jwt_validation field on the
envoy.filters.network.reverse_tunnelfilter. When configured, the bearer token carried in the handshake request is verified (signature, issuer, audiences, andexp) against an inlinelocal_jwksbefore the connection is accepted and its socket registered, so a forged or expired token cannot establish a usable reverse tunnel. Ajwt_validationblock requires anissuer, and tokens without anexpclaim are rejected. Verified claims are published as dynamic metadata so the existingvalidationblock can bind a claimed identifier to a verified claim via%DYNAMIC_METADATA(namespace:claim)%. Only inline JWKS (synchronous verification) is supported; remote JWKS fetching is not yet implemented.reverse_tunnel: The downstream reverse-tunnel initiator (
envoy.bootstrap.reverse_tunnel.downstream_socket_interface) now includes two additional identifiers in the HTTP handshake it sends to the acceptor:x-envoy-reverse-tunnel-worker-id(the initiator worker dispatcher name, e.g.worker_2) andx-envoy-reverse-tunnel-connection-id(the initiator’s per-connection id). Both are surfaced in the initiator access log via the newworker_idandconnection_idfields of theenvoy.reverse_tunnel.initiatordynamic metadata namespace. The upstream acceptor (envoy.bootstrap.reverse_tunnel.upstream_socket_interface) now parses these headers and exposes them on every reverse-tunnel lifecycle event as theinitiator_worker_idandinitiator_connection_idfields of theenvoy.reverse_tunnel.lifecycledynamic metadata namespace and as theenvoy.reverse_tunnel.initiator_worker_id/envoy.reverse_tunnel.initiator_connection_idconnection filter-state keys. Together these let tunnels originating from different workers/connections of the same initiator be told apart and correlated across both ends.stats: Added per-cluster and per-listener
stats_matcherconfiguration that overrides the bootstrap stats_config matcher for the specific cluster or listener. When this field is configured, legacyenvoy.stats_matchermetadata is ignored.tracing: tracing: added set_instrumentation_scope option to the OpenTelemetry tracer to allow controlling the emission of the instrumentation scope name and version in traces.
watchdog: Added envoy.watchdog.backtrace_action, a new watchdog action that logs a stack backtrace of stuck threads when the watchdog fires. A configurable cooldown prevents duplicate backtraces for the same thread.