[Bugfix][Frontend] Fix batch chat endpoint corrupting logprobs when return_token_ids is set (#47384)

Signed-off-by: David Feng <fenghourun@meta.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
David Feng
2026-07-03 03:01:34 -07:00
committed by GitHub
co-authored by Claude
parent 400a9c386d
commit 18f658bb31
3 changed files with 80 additions and 1 deletions
@@ -258,7 +258,7 @@ class OpenAIServingChatBatch(OpenAIServingChat):
top_logprobs=output.logprobs,
num_output_top_logprobs=request.top_logprobs,
tokenizer=tokenizer,
return_as_token_id=request.return_token_ids,
return_as_token_id=request.return_tokens_as_token_ids,
)
else:
logprobs = None