[Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
This commit is contained in:
@@ -757,7 +757,11 @@ class OpenAIServingChat(GenerateBaseServing):
|
||||
# finish_reason is:
|
||||
# "tool_calls" for "auto" or "required" tool calls,
|
||||
# and "stop" for named tool calls.
|
||||
if tools_streamed[i] and not tool_choice_function_name:
|
||||
if (
|
||||
tools_streamed[i]
|
||||
and not tool_choice_function_name
|
||||
and output.finish_reason == "stop"
|
||||
):
|
||||
finish_reason_ = "tool_calls"
|
||||
else:
|
||||
finish_reason_ = (
|
||||
|
||||
Reference in New Issue
Block a user