AI token usage (proto)

Warning

This API feature is currently work-in-progress. API features marked as work-in-progress are not considered stable, are not covered by the threat model, are not supported by the security team, and are subject to breaking changes. Do not use this feature without understanding each of the previous points.

data.ai.v3.TokenUsage

[data.ai.v3.TokenUsage proto]

Canonical LLM token usage extracted from a provider response by the AI Protocol Manager filter, published as typed dynamic metadata under one namespace at a clean end of stream.

Counts are normalized onto one inclusive contract regardless of wire API: input_tokens covers all input consumed (uncached input, cached reads, cache writes, and tool-use prompt tokens) and output_tokens covers all generated output including reasoning/thought tokens; the detail messages are subsets of those canonical values. Every value remains provider-reported and is not independently verified by Envoy.

Every count is optional: wire APIs report different subsets, and a status-only record (api_protocol, model when captured, and extraction_status: FAILED) is published when extraction failed outright, distinguishing that from a response that supplied no usage (which publishes nothing).

{
  "api_protocol": ...,
  "model": ...,
  "input_tokens": {...},
  "output_tokens": {...},
  "total_tokens": {...},
  "input_token_details": {...},
  "output_token_details": {...},
  "provider_total_tokens": {...},
  "extraction_status": ...
}

api_protocol: (type.ai.v3.ApiProtocol) The wire API the usage was extracted from. This names an API contract, not a provider identity: any OpenAI-compatible backend reports an OpenAI protocol here.

model: (string) The model reported by the response, when present.

input_tokens: (UInt64Value) Canonical inclusive input token count.

output_tokens: (UInt64Value) Canonical inclusive output token count.

total_tokens: (UInt64Value) Canonical total: input_tokens + output_tokens. Present only when both canonical components are known, so the triple is always internally consistent. Compare against provider_total_tokens to detect a provider that reports inconsistently or in buckets unknown to the extractor.

input_token_details: (data.ai.v3.InputTokenDetails) Canonical input token breakdown.

output_token_details: (data.ai.v3.OutputTokenDetails) Canonical output token breakdown.

provider_total_tokens: (UInt64Value) The total token count reported directly by the provider, when it reported one — preserved regardless of whether it agrees with total_tokens.

extraction_status: (data.ai.v3.TokenUsage.ExtractionStatus) Quality of the extraction result.

Enum data.ai.v3.TokenUsage.ExtractionStatus

[data.ai.v3.TokenUsage.ExtractionStatus proto]

Quality of the extraction result.

EXTRACTION_STATUS_UNSPECIFIED: (DEFAULT) ⁣

COMPLETE: ⁣All observed usage-bearing data was parsed successfully.

PARTIAL: ⁣Usable counts were extracted, but some usage-bearing input was lost: malformed, truncated, or skipped by a configured resource limit. The counts may be stale (an earlier cumulative snapshot) or incomplete.

FAILED: ⁣Usage-bearing input was observed but no usable canonical count could be extracted; only api_protocol and model may be populated.

data.ai.v3.InputTokenDetails

[data.ai.v3.InputTokenDetails proto]

Subsets of input_tokens.

{
  "cached_tokens": {...},
  "cache_creation_tokens": {...},
  "tool_use_tokens": {...}
}

cached_tokens: (UInt64Value) Cache-read input tokens.

cache_creation_tokens: (UInt64Value) Cache-creation (cache-write) input tokens.

tool_use_tokens: (UInt64Value) Tool-use prompt tokens.

data.ai.v3.OutputTokenDetails

[data.ai.v3.OutputTokenDetails proto]

Subsets of output_tokens.

{
  "reasoning_tokens": {...}
}

reasoning_tokens: (UInt64Value) Reasoning or thought tokens.