[Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958)

Signed-off-by: sfeng33 <4florafeng@gmail.com>
This commit is contained in:
Flora Feng
2026-09-28 04:12:04 +00:00
committed by GitHub
parent 4a190dc518
commit 6f65fde419
3 changed files with 84 additions and 17 deletions
@@ -91,6 +91,7 @@ class OpenAIServingChatBatch(OpenAIServingChat):
for messages in request.messages:
single_request = request.to_chat_completion_request(messages)
if renderer.use_harmony:
renderer.adjust_harmony_request(single_request)
conversation, engine_prompts = renderer._make_request_with_harmony(
single_request, should_include_tools=tool_dicts is not None
)