Kafka Mesh (proto)
This extension has the qualified name envoy.filters.network.kafka_mesh
Note
This extension is only available in contrib images.
Note
This extension is work-in-progress. Functionality is incomplete and it is not intended for production use.
This extension is not hardened and should only be used in deployments where both the downstream and upstream are trusted.
Tip
This extension extends and can be used with the following extension category:
Warning
This API feature is currently work-in-progress. API features marked as work-in-progress are not considered stable, are not covered by the threat model, are not supported by the security team, and are subject to breaking changes. Do not use this feature without understanding each of the previous points.
Kafka Mesh configuration overview.
extensions.filters.network.kafka_mesh.v3alpha.KafkaMesh
[extensions.filters.network.kafka_mesh.v3alpha.KafkaMesh proto]
{
"advertised_host": ...,
"advertised_port": ...,
"upstream_clusters": [],
"forwarding_rules": [],
"consumer_proxy_mode": ...
}
- advertised_host
(string, REQUIRED) Envoy’s host that’s advertised to clients. Has the same meaning as corresponding Kafka broker properties. Usually equal to filter chain’s listener config, but needs to be reachable by clients (so 0.0.0.0 will not work).
- advertised_port
(int32) Envoy’s port that’s advertised to clients.
- upstream_clusters
(repeated extensions.filters.network.kafka_mesh.v3alpha.KafkaClusterDefinition) Upstream clusters this filter will connect to.
- forwarding_rules
(repeated extensions.filters.network.kafka_mesh.v3alpha.ForwardingRule) Rules that will decide which cluster gets which request.
- consumer_proxy_mode
(extensions.filters.network.kafka_mesh.v3alpha.KafkaMesh.ConsumerProxyMode) How the consumer proxying should behave - this relates mostly to Fetch request handling.
Enum extensions.filters.network.kafka_mesh.v3alpha.KafkaMesh.ConsumerProxyMode
[extensions.filters.network.kafka_mesh.v3alpha.KafkaMesh.ConsumerProxyMode proto]
- StatefulConsumerProxy
(DEFAULT) Records received are going to be distributed amongst downstream consumer connections. In this mode Envoy uses librdkafka consumers pointing at upstream Kafka clusters, what means that these consumers’ position is meaningful and affects what records are received from upstream. Users might want to take a look into these consumers’ custom configuration to manage their auto-committing capabilities, as it will impact Envoy’s behaviour in case of restarts.
extensions.filters.network.kafka_mesh.v3alpha.KafkaClusterDefinition
[extensions.filters.network.kafka_mesh.v3alpha.KafkaClusterDefinition proto]
{
"cluster_name": ...,
"bootstrap_servers": ...,
"partition_count": ...,
"producer_config": {...},
"consumer_config": {...}
}
- cluster_name
(string, REQUIRED) Cluster name.
- bootstrap_servers
(string, REQUIRED) Kafka cluster address.
- partition_count
(int32) Default number of partitions present in this cluster. This is especially important for clients that do not specify partition in their payloads and depend on this value for hashing. The same number of partitions is going to be used by upstream-pointing Kafka consumers for consumer proxying scenarios.
extensions.filters.network.kafka_mesh.v3alpha.ForwardingRule
[extensions.filters.network.kafka_mesh.v3alpha.ForwardingRule proto]
{
"target_cluster": ...,
"topic_prefix": ...
}
- target_cluster
(string) Cluster name.
- topic_prefix
(string) Intended place for future types of forwarding rules.