[Frontend] Add return_mm_kwargs to render requests (#60728)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -458,6 +458,18 @@ class ChatCompletionRequest(OpenAIBaseModel):
|
||||
),
|
||||
)
|
||||
|
||||
return_mm_kwargs: bool = Field(
|
||||
default=True,
|
||||
description=(
|
||||
"If false, the render response's `features` set `kwargs_data` "
|
||||
"and `mm_metadata` to null, for callers that need only the token "
|
||||
"layout and item hashes, such as cache-aware routers. Do not send "
|
||||
"such a response to `/inference/v1/generate`, which reads a null "
|
||||
"`kwargs_data` as every item being cached. Only supported on the "
|
||||
"render endpoints; ignored on regular generation endpoints."
|
||||
),
|
||||
)
|
||||
|
||||
cache_salt: str | None = Field(
|
||||
default=None,
|
||||
min_length=1,
|
||||
|
||||
Reference in New Issue
Block a user