[Bugfix][Frontend] Fix batch chat endpoint corrupting logprobs when return_token_ids is set (#47384)
Signed-off-by: David Feng <fenghourun@meta.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -258,7 +258,7 @@ class OpenAIServingChatBatch(OpenAIServingChat):
|
||||
top_logprobs=output.logprobs,
|
||||
num_output_top_logprobs=request.top_logprobs,
|
||||
tokenizer=tokenizer,
|
||||
return_as_token_id=request.return_token_ids,
|
||||
return_as_token_id=request.return_tokens_as_token_ids,
|
||||
)
|
||||
else:
|
||||
logprobs = None
|
||||
|
||||
Reference in New Issue
Block a user