http: add sse_to_metadata filter for stream parsing

[PeterL328]

Commit Message: Add sse_to_metadata HTTP filter with pluggable content parsers
Additional Description:
This PR introduces a new HTTP filter for extracting values from Server-Sent Events (SSE) response streams and writing them into dynamic metadata. The filter uses a pluggable content parser architecture that separates SSE protocol handling from content-specific parsing logic, making it extensible for future parser types (XML, plaintext, regex, etc.).

The primary use case is extracting token usage information from LLM streaming responses (e.g., OpenAI, Anthropic) for logging, observability and rate-limit token reporting.

Key Architecture

• Filter Layer (envoy.filters.http.sse_to_metadata): Handles SSE protocol parsing, buffering, and metadata writing
• Parser Interface (envoy.sse_content_parser): Generic interface for parsing SSE event data fields
• JSON Parser (envoy.sse_content_parsers.json): First implementation supporting JSON path-based extraction with on_present/on_missing/on_error semantics

This layered design allows future parsers (XML, plaintext, regex) to be added without modifying the filter.

Key Features

• SSE Protocol Compliance: https://html.spec.whatwg.org/multipage/server-sent-events.html including:
  • CRLF/CR/LF line endings
  • Comment handling (: comment lines)
  • Multiple data fields per event
  • Chunked transfer encoding
  • Event buffering with configurable size limits
• JSON Content Parser:
  • Path-based value extraction (e.g., ["usage", "total_tokens"])
  • Type conversion (STRING, NUMBER, PROTOBUF_VALUE)
  • Conditional metadata writing: on_present, on_missing, on_error
  • Hardcoded fallback values for error conditions
• Pluggable Architecture:
  • TypedExtensionConfig for parser selection
  • Factory pattern for per-request parser instances
  • Clean interface separation (filter agnostic to parser internals)
  • Easy to add new parsers (XML, plaintext, regex, etc.)
• Multiple Metadata Targets:
  • Write to multiple namespaces per rule
  • Preserve existing metadata option
  • Per-namespace dynamic metadata
• Protection Mechanisms:
  • Configurable max_event_size (default: 8KB, max: 10MB)
  • Prevents unbounded buffer growth from malformed streams
  • Content-Type validation (text/event-stream)
• Flexible Matching:
  • stop_processing_on_match: Choose first or last occurrence
  • Per-rule configuration
  • Deferred execution for fallback values

Risk Level: Low (new filter)
Testing:

• Filter unit and integration tests
• SSE parser unit and fuzzer test
• JSON content parser unit test

Docs Changes: Added comprehensive documentation for sse_to_metadata filter and JSON content parser
Release Notes: http: Added envoy.filters.http.sse_to_metadata filter for extracting values from Server-Sent Events (SSE) streams and writing them to dynamic metadata. Supports pluggable content parsers with initial JSON parser implementation. Useful for token-based rate limiting, logging and observability of LLM streaming APIs.
Platform Specific Features: N/A

Fixes #29758

[repokitteh-read-only]

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @adisuissa
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).

🐱

Caused by: #42762 was opened by PeterL328.

see: more, trace.

[PeterL328]

Tagging @JuniorHsu and @KBaichoo for review 🙏
Seems we need at least 2 owners and 1 maintainer. Wonder if can add you two?

[JuniorHsu]

Tagging @JuniorHsu and @KBaichoo for review 🙏 Seems we need at least 2 owners and 1 maintainer. Wonder if can add you two?

happy to help review and be one of the owner :)

[PeterL328]

/retest

[PeterL328]

@KBaichoo added you as maintainer for now but let me know if we should change that 😁

[PeterL328]

@adisuissa Looking for some guidance here please
Seems CI is failing due to proto_sync wants to delete api/envoy/extensions/sse_content_parsers/json/v3/* which is the new extension category. Are there some steps required for new extension category that I am missing? seems CI just wants to remove it.

[adisuissa]

@adisuissa Looking for some guidance here please Seems CI is failing due to proto_sync wants to delete api/envoy/extensions/sse_content_parsers/json/v3/* which is the new extension category. Are there some steps required for new extension category that I am missing? seems CI just wants to remove it.

It's been a while since I've seen this, but the script may require the extension to be referenced (so maybe if the doc error is fixed, then it will keep it).
You can also look at other PRs that added similar types, and see if there was something special there.

[PeterL328]

/retest

[botengyao]

@PeterL328, wondering if you are willing to split the SSE parser out to another PR, and I will take a look, but up to you.

[PeterL328]

wondering if you are willing to split the SSE parser out to another PR, and I will take a look, but up to you.

@botengyao sure i can do that. I'll move the common/sse part and related tests (unit + fuzzer) to another PR. This PR is getting larger so a smaller one will be better to review anyways 😄

[PeterL328]

Created SSE parser util PR
#43081
cc: @botengyao

After SSE parser diff lands, I will remove the related files/changes in this diff.

[PeterL328]

@adisuissa looks like CI issue relating to proto are fixed.
Could you please take a look at new API changes. Ty 🙏

[adisuissa]

Please add a reference to preserve_existing_metadata_value.

[wbpcode]

At leave this should be under /api/envoy/extensions/http/sse_content_parsers/....

[PeterL328]

sg

[adisuissa]

2 high-level questions:

1. is this field part of the json-content parsing (what I consider as output) or should it be part of the SSE parsing (what I consider as input)? - My first thought was this is related to the matched rules on the json-content parsing side, but the phrase in the comment ("extracting a model name from the first sse event") implied that it is related to the SSE events rather than the matched rules.
2. Thinking out loud: I'm no expert in the use-cases of this, but I wonder if it makes sense to configure a number here (stop_processing_after_matches) which will be optional to set (WKT type). - Note that I'm not saying to go this way, I just want to make sure that this was considered, and if needed in the future, then could be done now.

[PeterL328]

Thanks for feedback.

1. For SSE-to-metadata filter is designed to be content agnostic, and stopping criteria should be purely based on content so let's update the comment here to be better here.

2. I thought about this a bit. I have two thoughts here:
   First I think I agree with you that even though most cases I would imagine would want to extract the first occurrence or last occurrence but we should keep API flexible in case we want extract after n occurrence. Let's go with something like UInt32Value stop_processing_after_matches where 0 means last, 1 means first and later N.
   Second I feel it's confusing now that stop_processing_on_first_match is applied to all the rules. Previously when we didn't use j2m rules we had per rule stop_processing_on_first_match which is more fine grained and clearer. So I think we can add a wrapper around j2m rule but have a stop_processing_after_matches on each rule.

[wbpcode]

Thanks for this contribution. To be honest, I think this could be supported by add SSE support at transform filter. For the AI token extraction, it should be easier. But anyway, this have got sponsor, so I will not block this if you guys do want this. It's not big deal to have overlap between different filters. :)

[wbpcode]

At leave this should be under /api/envoy/extensions/http/sse_content_parsers/....

[wbpcode]

/wait for API review