[Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583)

Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com>
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
This commit is contained in:
Denis Ermakov
2026-09-25 16:33:19 +00:00
committed by GitHub
co-authored by Yanxiao Zhao Prakhar Agarwal Flora Feng
parent 6491f481a7
commit f339e0e757
2 changed files with 72 additions and 4 deletions
@@ -693,11 +693,13 @@ class OpenAIServingChat(GenerateBaseServing):
logprobs = None
if delta_message is None:
# NOTE: If return_token_ids is enabled, we still need to
# send a chunk with token_ids even if delta_message is None
# NOTE: If return_token_ids or logprobs are enabled, we
# still need to send a chunk even if delta_message is None
# to ensure all tokens are included in the response
if output.finish_reason is None and (
not request.return_token_ids or hide_stream_metadata
if (
output.finish_reason is None
and logprobs is None
and (not request.return_token_ids or hide_stream_metadata)
):
continue
delta_message = DeltaMessage()