.. _arch_overview_dynamic_modules: Dynamic modules =============== .. attention:: The dynamic modules feature is currently under active development. Capabilities will be expanded over time and it still lacks some features that are available in other extension mechanisms. We are looking for feedback from the community to improve the feature. Envoy has support for loading shared libraries at runtime to extend its functionality. In Envoy, these are known as "dynamic modules." More specifically, dynamic modules are shared libraries that implement the :repo:`ABI ` written in a pure C header file. The ABI defines a set of functions that the dynamic module must implement to be loaded by Envoy. Also, it specifies the functions implemented by Envoy that the dynamic module can call to interact with Envoy. Implementing the ABI from scratch requires an extensive understanding of the Envoy internals. For users, we provide an official SDK that abstracts these details and provides a high-level API to implement dynamic modules. The SDK is currently available in C++, Go, and Rust. In theory, any language that can produce a shared library can be used to implement dynamic modules. Future development may include support for other languages. Currently, dynamic modules are supported at the following extension points: * As a :ref:`bootstrap extension `. * As a :ref:`cluster `. * As a :ref:`listener filter `. * As a :ref:`UDP listener filter `. * As an :ref:`access logger `. * As a :ref:`formatter `. * As a :ref:`stats sink `. * As a :ref:`network filter `. * As an :ref:`HTTP filter `. * As an :ref:`HTTP matching data input `. * As an :ref:`input matcher `. * As a :ref:`TLS certificate validator `. * As a :ref:`transport socket `. * As a :ref:`load balancing policy `. * As an :ref:`upstream HTTP TCP bridge `. * As a :ref:`tracer `. * As a :ref:`health checker `. There are a few design goals for the dynamic modules: 1. **Performance**: The dynamic modules should have minimal overhead compared to the built-in C++ extensions. For example, the dynamic modules are able to access HTTP headers as well as body without copying them unlike any other extension mechanisms. 2. **Ease of Use**: The SDK should provide a high-level API that abstracts the details of the Envoy internals. 3. **Flexibility**: The dynamic modules should be able to implement any functionality that can be implemented by the built-in C++ extensions without performance penalty. This is work in progress and many features are not yet available. Compatibility -------------------------- Since a dynamic module is loaded at runtime, it must be ABI-compatible with the Envoy binary that loads it. Envoy's dynamic modules have stricter compatibility requirements than Envoy's other extension mechanisms, such as Lua, Wasm or External Processor. Stabilizing the ABI is challenging due to the way the ABI needs to be tightly coupled to Envoy's internals. Currently, we guarantee **forward compatibility within one version**: a dynamic module built with the SDK for Envoy version X.Y will work with Envoy versions X.Y and X.(Y+1). Breaking changes to the ABI may occur in later versions. To ensure compatibility, it is recommended to rebuild your dynamic modules with the SDK matching your target Envoy version in a timely manner. Module discovery -------------------------- A dynamic module is referenced by its name as in the :ref:`configuration API `. The name is used to search for the shared library file in the search path. The search path is configured by the environment variable ``ENVOY_DYNAMIC_MODULES_SEARCH_PATH``. The actual search path is ``${ENVOY_DYNAMIC_MODULES_SEARCH_PATH}/lib${name}.so``. If the environment variable is not set, the current working directory is used instead. After searching in the specified search path, the standard library paths such as ``LD_LIBRARY_PATH`` and ``/usr/lib`` are searched as well following the behavior of ``dlopen(3)``. For example, when the name ``my_module`` is referenced in the configuration and ``ENVOY_DYNAMIC_MODULES_SEARCH_PATH`` is set to ``/path/to/modules``, Envoy will first look for ``/path/to/modules/libmy_module.so``, then ``$LD_LIBRARY_PATH/libmy_module.so``, and finally ``/usr/lib/libmy_module.so``, etc. Safety -------------------------- The dynamic modules should be used under the assumption that all modules are fully trusted and have the same privilege level as the main Envoy program. Since these modules run in the same process as Envoy, they can access all memory and resources available to the main process. This makes it unfeasible to enforce security boundaries between Envoy and the modules, as they share the same address space and permissions. It is essential that any dynamic module undergo thorough testing and validation before deployment just like any other application code. Error handling (Rust SDK) -------------------------- The Rust SDK provides an optional ``CatchUnwind`` wrapper that can be used to wrap filter implementations. When a wrapped callback panics, the SDK logs the panic payload and returns a fail-closed default: * HTTP request-path callbacks send a 500 response and return ``StopIteration``. * HTTP response-path callbacks reset the stream and return ``StopIteration``. * Network filter callbacks close the connection and return ``StopIteration``. * Listener filter callbacks close the socket and return ``StopIteration``. The SDK always guards each callback at the FFI boundary so a panic can never unwind into Envoy and corrupt the process, regardless of whether ``CatchUnwind`` is used. The wrapper adds graceful filter-level teardown on top of that guard. The affected request or connection is terminated. Other traffic is unaffected. Getting started -------------------------- We have a dedicated repository for the dynamic module examples to help you get started. The repository is available at `envoyproxy/dynamic-modules-examples `_ Statistics --------------------------- All dynamic-module extension types emit the following statistics in the shared ``dynamic_modules.`` namespace. These stats track failures encountered while loading the extension's configuration. Each one is tagged with ``config_name``, set to the configured name of the dynamic-module extension instance — for example the :ref:`filter_name ` for the HTTP filter, ``transport_socket_name`` for the transport socket, ``lb_policy_name`` for the load-balancing policy, ``tracer_name`` for the tracer or ``cluster_name`` for the cluster (``default`` if the extension has no per-instance name, as for the UDP listener filter). .. csv-table:: :header: Name, Type, Description :widths: 1, 1, 2 module_load_error, Counter, "Total dynamic modules that could not be loaded (missing or invalid module source, ``dlopen`` failure, by-name lookup miss, or a required ABI symbol could not be resolved)." config_init_error, Counter, "Total configurations that failed to initialize after the module loaded successfully (the module rejected or failed to parse the supplied configuration)." remote_fetch_error, Counter, "Total failures fetching or loading a remote module source, including rejected cache misses when ``nack_on_cache_miss`` is set. Only the HTTP filter supports remote module sources." per_route_config_error, Counter, "Total per-route configurations that failed to load or initialize. Only emitted by the HTTP filter." In addition to the counters above, a module may define its own custom metrics. These are emitted under the configurable :ref:`metrics_namespace ` (``dynamicmodulescustom`` by default), separately from the ``dynamic_modules.`` namespace above.