[Bugfix][Parser] Pass token IDs to parser.parse() in Responses API and batch serving (#46843)
Signed-off-by: Ben Browning <bbrownin@redhat.com>
This commit is contained in:
@@ -267,6 +267,7 @@ class OpenAIServingChatBatch(OpenAIServingChat):
|
||||
reasoning, content, _ = parser.parse(
|
||||
output.text,
|
||||
request=request, # type: ignore[arg-type]
|
||||
model_output_token_ids=output.token_ids,
|
||||
)
|
||||
if not request.include_reasoning:
|
||||
reasoning = None
|
||||
|
||||
Reference in New Issue
Block a user