[responsesAPI][6] Fix multi turn MCP tokenization (#30230)

Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
This commit is contained in:
Andrew Xia
2025-12-10 10:13:13 +08:00
committed by GitHub
co-authored by Andrew Xia
parent abe93bce59
commit c3487aca34
5 changed files with 110 additions and 13 deletions
@@ -1339,6 +1339,7 @@ class OpenAIServing:
)
engine_prompt = engine_prompts[0]
request_prompt = request_prompts[0]
prompt_text, _, _ = self._get_prompt_components(request_prompt)
# Update the sampling params.
sampling_params.max_tokens = self.max_model_len - len(