[Bugfix] fix DP-aware routing in OpenAI API requests (#29002)

Signed-off-by: inkcherry <mingzhi.liu@amd.com>
This commit is contained in:
inkcherry
2025-12-18 09:50:42 -08:00
committed by GitHub
parent 686cbaac64
commit 500f26e6d3
7 changed files with 68 additions and 0 deletions
@@ -230,6 +230,7 @@ class OpenAIServingCompletion(OpenAIServing):
lora_request=lora_request,
trace_headers=trace_headers,
priority=request.priority,
data_parallel_rank=data_parallel_rank,
)
generator = self.engine_client.generate(