[Frontend] Add request_id to the Request object so they can be controlled better via external load balancers (#21009)

Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
This commit is contained in:
kourosh hakhamaneshi
2025-07-25 06:49:11 -07:00
committed by GitHub
parent 29c6fbe58c
commit 9fe98d4250
3 changed files with 27 additions and 3 deletions
+3 -2
View File
@@ -163,8 +163,9 @@ class OpenAIServingEmbedding(EmbeddingMixin):
for the API specification. This API mimics the OpenAI Embedding API.
"""
model_name = self._get_model_name(request.model)
request_id = (f"{self.request_id_prefix}-"
f"{self._base_request_id(raw_request)}")
request_id = (
f"{self.request_id_prefix}-"
f"{self._base_request_id(raw_request, request.request_id)}")
ctx = EmbeddingServeContext(
request=request,