1.20.0 (October 5, 2021) ======================== Incompatible Behavior Changes ----------------------------- *Changes that are expected to cause an incompatibility if applicable; deployment changes are likely required* * config: due to the switch to using work-in-progress annotations and warnings to indicate APIs subject to change, the following API packages have been force migrated from ``v3alpha`` to ``v3``: ``envoy.extensions.access_loggers.open_telemetry.v3``, ``envoy.extensions.cache.simple_http_cache.v3``, ``envoy.extensions.filters.http.admission_control.v3``, ``envoy.extensions.filters.http.bandwidth_limit.v3``, ``envoy.extensions.filters.http.cache.v3``, ``envoy.extensions.filters.http.cdn_loop.v3``, ``envoy.extensions.filters.http.ext_proc.v3``, ``envoy.extensions.filters.http.oauth2.v3``, ``envoy.extensions.filters.network.sni_dynamic_forward_proxy.v3``, ``envoy.extensions.filters.udp.dns_filter.v3``, ``envoy.extensions.transport_sockets.s2a.v3``, ``envoy.extensions.watchdog.profile_action.v3``, ``envoy.service.ext_proc.v3``, and ``envoy.watchdog.v3``. If your production deployment was using one of these APIs, you will be forced to potentially vendor the old proto file to continue serving old versions of Envoy. The project realizes this is unfortunate because some of these are known to be used in production, however the project does not have the resources to undergo a migration in which we support ``v3alpha`` and ``v3`` at the same time. The switch to using work-in-progress annotations with clear and explicit warnings will avoid any such issue in the future. We apologize again for any difficulty this change causes, though it is for the best. Additionally, some of the above namespaces have had their work-in-progress annotations removed due to known production usage. Thus, they will not warn and are offered full API stability support by the project from this point forward. * config: the ``--bootstrap-version`` CLI flag has been removed, Envoy has only been able to accept v3 bootstrap configurations since 1.18.0. * contrib: the :ref:`squash filter ` has been moved to :ref:`contrib images `. * contrib: the :ref:`kafka broker filter ` has been moved to :ref:`contrib images `. * contrib: the :ref:`RocketMQ proxy filter ` has been moved to :ref:`contrib images `. * contrib: the :ref:`Postgres proxy filter ` has been moved to :ref:`contrib images `. * contrib: the :ref:`MySQL proxy filter ` has been moved to :ref:`contrib images `. * dns_filter: :ref:`dns_filter ` protobuf fields have been renumbered to restore compatibility with Envoy 1.18, breaking compatibility with Envoy 1.19.0 and 1.19.1. The new field numbering allows control planes supporting Envoy 1.18 to gracefully upgrade to :ref:`dns_resolution_config `, provided they skip over Envoy 1.19.0 and 1.19.1. Control planes upgrading from Envoy 1.19.0 and 1.19.1 will need to vendor the corresponding protobuf definitions to ensure that the renumbered fields have the types expected by those releases. * extensions: deprecated extension names now default to triggering a configuration error. The previous warning-only behavior may be temporarily reverted by setting the runtime key ``envoy.deprecated_features.allow_deprecated_extension_names`` to true. Minor Behavior Changes ---------------------- *Changes that may cause incompatibilities for some users, but should not for most* * client_ssl_auth filter: now sets additional termination details and ``UAEX`` response flag when the client certificate is not in the allowed-list. * config: configuration files ending in .yml now load as YAML. * config: configuration file extensions now ignore case when deciding the file type. E.g., .JSON files load as JSON. * config: reduced log level for "Unable to establish new stream" xDS logs to debug. The log level for "gRPC config stream closed" is now reduced to debug when the status is ``Ok`` or has been retriable (``DeadlineExceeded``, ``ResourceExhausted``, or ``Unavailable``) for less than 30 seconds. * config: use of work-in-progress API files, messages, or fields will now generate an explicit warning. Please read the text about ``(xds.annotations.v3.file_status).work_in_progress``, ``(xds.annotations.v3.message_status).work_in_progress``, and ``(xds.annotations.v3.field_status).work_in_progress`` `here `_ for more information. Some APIs that are known to be implicitly not work-in-progress have been force migrated and are individually indicated elsewhere in the release notes. A server-wide ``wip_protos`` counter has also been added in :ref:`server statistics ` to track this. * ext_authz: fixed skipping authentication when returning either a direct response or a redirect. This behavior can be temporarily reverted by setting the ``envoy.reloadable_features.http_ext_authz_do_not_skip_direct_response_and_redirect`` runtime guard to false. * grpc: gRPC async client can be cached and shared across filter instances in the same thread, this feature is turned off by default, can be turned on by setting runtime guard ``envoy.reloadable_features.enable_grpc_async_client_cache`` to true. * http: correct the use of the ``x-forwarded-proto`` header and the ``:scheme`` header. Where they differ (which is rare) ``:scheme`` will now be used for serving redirect URIs and cached content. This behavior can be reverted by setting runtime guard ``correct_scheme_and_xfp`` to false. * http: reject requests with #fragment in the URI path. The fragment is not allowed to be part of the request URI according to RFC3986 (3.5), RFC7230 (5.1) and RFC 7540 (8.1.2.3). Rejection of requests can be changed to stripping the #fragment instead by setting the runtime guard ``envoy.reloadable_features.http_reject_path_with_fragment`` to false. This behavior can further be changed to the deprecated behavior of keeping the fragment by setting the runtime guard ``envoy.reloadable_features.http_strip_fragment_from_path_unsafe_if_disabled``. This runtime guard must only be set to false when existing non-compliant traffic relies on #fragment in URI. When this option is enabled, Envoy request authorization extensions may be bypassed. This override and its associated behavior will be decommissioned after the standard deprecation period. * http: set the default :ref:`lazy headermap threshold ` to 3, which defines the minimal number of headers in a request/response/trailers required for using a dictionary in addition to the list. Setting the ``envoy.http.headermap.lazy_map_min_size`` runtime feature to a non-negative number will override the default value. * http: stop processing pending H/2 frames if connection transitioned to a closed state. This behavior can be temporarily reverted by setting the ``envoy.reloadable_features.skip_dispatching_frames_for_closed_connection`` to false. * listener: added the :ref:`enable_reuse_port ` field and changed the default for ``reuse_port`` from false to true, as the feature is now well supported on the majority of production Linux kernels in use. The default change is aware of the hot restart, as otherwise, the change would not be backward compatible between restarts. This means that hot restarting onto a new binary will retain the default of false until the binary undergoes a full restart. To retain the previous behavior, either explicitly set the new configuration field to false, or set the runtime feature flag ``envoy.reloadable_features.listener_reuse_port_default_enabled`` to false. As part of this change, the use of ``reuse_port`` for TCP listeners on both macOS and Windows has been disabled due to suboptimal behavior. See the field documentation for more information. * listener: destroy per network filter chain stats when a network filter chain is removed during the listener in-place update. * quic: enables IETF connection migration. This feature requires a stable UDP packet routine in the L4 load balancer with the same first-4-bytes in connection id. It can be turned off by setting runtime guard ``envoy.reloadable_features.FLAGS_quic_reloadable_flag_quic_connection_migration_use_new_cid_v2`` to false. * thrift_proxy: allow Framed and Header transport combinations to perform :ref:`payload passthrough `. Bug Fixes --------- *Changes expected to improve the state of the world and are unlikely to have negative effects* * access log: fix ``%UPSTREAM_CLUSTER%`` when used in http upstream access logs. Previously, it was always logging as an unset value. * aws request signer: fix the AWS Request Signer extension to correctly normalize the path and query string to be signed according to AWS' guidelines, so that the hash on the server side matches. See `AWS SigV4 documentation `_. * cluster: delete pools when they're idle to fix unbounded memory use when using PROXY protocol upstream with tcp_proxy. This behavior can be temporarily reverted by setting the ``envoy.reloadable_features.conn_pool_delete_when_idle`` runtime guard to false. * cluster: finish cluster warming even if hosts are removed before health check initialization. This only affected clusters with :ref:`ignore_health_on_host_removal `. * compressor: fix a bug where if trailers were added and a subsequent filter paused the filter chain, the request could be stalled. This behavior can be reverted by setting ``envoy.reloadable_features.fix_added_trailers`` to false. * dynamic forward proxy: fixing a validation bug where san and sni checks were not applied setting :ref:`http_protocol_options ` via :ref:`typed_extension_protocol_options `. * ext_authz: fix the ext_authz filter to correctly merge multiple same headers using the ',' as separator in the check request to the external authorization service. * ext_authz: fix the use of ``append`` field of :ref:`response_headers_to_add ` to set or append encoded response headers from a gRPC auth server. * ext_authz: fix the HTTP ext_authz filter to respond with ``403 Forbidden`` when a gRPC auth server sends a denied check response with an empty HTTP status code. * ext_authz: the network ext_authz filter now correctly sets dynamic metadata returned by the authorization service for non-OK responses. This behavior now matches the http ext_authz filter. * hcm: remove deprecation for :ref:`xff_num_trusted_hops ` and forbid mixing ip detection extensions with old related knobs. * http: limit use of deferred resets in the http2 codec to server-side connections. Use of deferred reset for client connections can result in incorrect behavior and performance problems. * listener: fixed an issue on Windows where connections are not handled by all worker threads. * lua: fix ``BodyBuffer`` setting a Lua string and printing Lua string containing hex characters. Previously, ``BodyBuffer`` setting a Lua string or printing strings with hex characters will be truncated. * xray: fix the AWS X-Ray tracer bug where span's error, fault and throttle information was not reported properly as per the `AWS X-Ray documentation `_. Before this fix, server error was reported under the 'annotations' section of the segment data. Removed Config or Runtime ------------------------- *Normally occurs at the end of the* :ref:`deprecation period ` * http: removed ``envoy.reloadable_features.http_upstream_wait_connect_response`` runtime guard and legacy code paths. * http: removed ``envoy.reloadable_features.allow_preconnect`` runtime guard and legacy code paths. * listener: removed ``envoy.reloadable_features.disable_tls_inspector_injection`` runtime guard and legacy code paths. * ocsp: removed ``envoy.reloadable_features.check_ocsp_policy deprecation`` runtime guard and legacy code paths. * ocsp: removed ``envoy.reloadable_features.require_ocsp_response_for_must_staple_certs deprecation`` and legacy code paths. * quic: removed ``envoy.reloadable_features.prefer_quic_kernel_bpf_packet_routing`` runtime guard. New Features ------------ * access_log: added :ref:`METADATA` token to handle all types of metadata (DYNAMIC, CLUSTER, ROUTE). * bootstrap: added :ref:`inline_headers ` in the bootstrap to make custom inline headers bootstrap configurable. * contrib: added new :ref:`contrib images ` which contain contrib extensions. * dns: added :ref:`V4_PREFERRED ` option to return V6 addresses only if V4 addresses are not available. * ext_authz: added :ref:`dynamic_metadata_from_headers ` to support emitting dynamic metadata from headers returned by an external authorization service via HTTP. * grpc reverse bridge: added a new :ref:`option ` to support streaming response bodies when withholding gRPC frames from the upstream. * grpc_json_transcoder: added support to unescape '+' in query parameters to space with a new config field :ref:`query_param_unescape_plus `. * http: added cluster_header in :ref:`weighted_clusters ` to allow routing to the weighted cluster specified in the request_header. * http: added :ref:`alternate_protocols_cache_options ` for enabling HTTP/3 connections to servers which advertise HTTP/3 support via `HTTP Alternative Services `_ and caching the advertisements to disk. * http: added :ref:`string_match ` in the header matcher. * http: added :ref:`x-envoy-upstream-stream-duration-ms ` that allows configuring the max stream duration via a request header. * http: added support for :ref:`max_requests_per_connection ` for both upstream and downstream connections. * http: sanitizing the referer header as documented :ref:`here `. This feature can be temporarily turned off by setting runtime guard ``envoy.reloadable_features.sanitize_http_header_referer`` to false. * http: validating outgoing HTTP/2 CONNECT requests to ensure that if ``:path`` is set that ``:protocol`` is present. This behavior can be temporarily turned off by setting runtime guard ``envoy.reloadable_features.validate_connect`` to false. * jwt_authn: added support for :ref:`Jwt Cache ` and its size can be specified by :ref:`jwt_cache_size `. * jwt_authn: added support for extracting JWTs from request cookies using :ref:`from_cookies `. * jwt_authn: added support for setting the extracted headers from a successfully verified JWT using :ref:`header_in_metadata ` to dynamic metadata. * listener: new listener metric ``downstream_cx_transport_socket_connect_timeout`` to track transport socket timeouts. * lua: added ``header:getAtIndex()`` and ``header:getNumValues()`` methods to :ref:`header object ` for retrieving the value of a header at certain index and get the total number of values for a given header. * matcher: added :ref:`invert ` for inverting the match result in the metadata matcher. * overload: add a new overload action that resets streams using a lot of memory. To enable the tracking of allocated bytes in buffers that a stream is using we need to configure the minimum threshold for tracking via :ref:`buffer_factory_config `. We have an overload action ``Envoy::Server::OverloadActionNameValues::ResetStreams`` that takes advantage of the tracking to reset the most expensive stream first. * rbac: added :ref:`destination_port_range ` for matching range of destination ports. * rbac: added :ref:`matcher` along with extension category ``extension_category_envoy.rbac.matchers`` for custom RBAC permission matchers. Added reference implementation for matchers :ref:`envoy.rbac.matchers.upstream_ip_port `. * route config: added :ref:`dynamic_metadata ` for routing based on dynamic metadata. * router: added retry options predicate extensions configured via :ref:`retry_options_predicates. ` These extensions allow modification of requests between retries at the router level. There are not currently any built-in extensions that implement this extension point. * router: added :ref:`per_try_idle_timeout ` timeout configuration. * router: added an optional :ref:`override_auto_sni_header ` to support setting SNI value from an arbitrary header other than host/authority. * sxg_filter: added filter to transform response to SXG package to :ref:`contrib images `. This can be enabled by setting :ref:`SXG ` configuration. * thrift_proxy: added support for :ref:`mirroring requests `. * udp: allows updating filter chain in-place through LDS, which is supported by Quic listener. Such listener config will be rejected in other connection-less UDP listener implementations. It can be reverted by ``envoy.reloadable_features.udp_listener_updates_filter_chain_in_place``. * udp: disallow L4 filter chain in config which configures connection-less UDP listener. It can be reverted by ``envoy.reloadable_features.udp_listener_updates_filter_chain_in_place``. * upstream: added support for :ref:`slow start mode `, which allows to progresively increase traffic for new endpoints. * upstream: extended :ref:`Round Robin load balancer configuration ` with :ref:`slow start ` support. * upstream: extended :ref:`Least Request load balancer configuration ` with :ref:`slow start ` support. * windows: added a new container image based on Windows Nanoserver 2022. * xray: request direction (``ingress`` or ``egress``) is recorded as X-Ray trace segment's annotation by name ``direction``. Deprecated ---------- * api: the :ref:`matcher ` field has been deprecated in favor of :ref:`matcher ` in order to break a build dependency. * cluster: :ref:`max_requests_per_connection ` is deprecated in favor of :ref:`max_requests_per_connection `. * http: the HeaderMatcher fields :ref:`exact_match `, :ref:`safe_regex_match `, :ref:`prefix_match `, :ref:`suffix_match ` and :ref:`contains_match ` are deprecated by :ref:`string_match `. * listener: :ref:`reuse_port ` has been deprecated in favor of :ref:`enable_reuse_port `. At the same time, the default has been changed from false to true. See above for more information.