[Frontend] Attach resolved logprobs to streaming derender chunks (#55029)

Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Shimi Bandiel
2026-09-29 18:43:44 +00:00
committed by GitHub
co-authored by Claude
parent df4dbe46e7
commit 4dc57d44c4
4 changed files with 224 additions and 8 deletions
@@ -546,6 +546,17 @@ class DerenderStreamState(BaseModel):
window is transiently empty (e.g. usage only final chunk).
"""
logprob_context_token_ids: list[int] = Field(default_factory=list, max_length=4)
"""Trailing sampled token IDs carried across chunks so byte-fallback
(U+FFFD) correction during logprob placeholder resolution has context
at chunk boundaries. Bounded to the 4-token window that
``_correct_decoded_token`` reads."""
logprob_text_offset: int = Field(default=0, ge=0)
"""Cumulative emitted text length. Seeds ``text_offset`` for completion
streaming logprobs so offsets stay absolute across chunks, mirroring
``initial_text_offset`` in the generate streaming path."""
output_token_ids: list[int] = Field(default_factory=list)
"""All output tokens seen so far. Parser path only.