[Frontend] Attach resolved logprobs to streaming derender chunks (#55029)
Signed-off-by: Shimi Bandiel <shimib@google.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -546,6 +546,17 @@ class DerenderStreamState(BaseModel):
|
||||
window is transiently empty (e.g. usage only final chunk).
|
||||
"""
|
||||
|
||||
logprob_context_token_ids: list[int] = Field(default_factory=list, max_length=4)
|
||||
"""Trailing sampled token IDs carried across chunks so byte-fallback
|
||||
(U+FFFD) correction during logprob placeholder resolution has context
|
||||
at chunk boundaries. Bounded to the 4-token window that
|
||||
``_correct_decoded_token`` reads."""
|
||||
|
||||
logprob_text_offset: int = Field(default=0, ge=0)
|
||||
"""Cumulative emitted text length. Seeds ``text_offset`` for completion
|
||||
streaming logprobs so offsets stay absolute across chunks, mirroring
|
||||
``initial_text_offset`` in the generate streaming path."""
|
||||
|
||||
output_token_ids: list[int] = Field(default_factory=list)
|
||||
"""All output tokens seen so far. Parser path only.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user