[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests (#61427)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
This commit is contained in:
Martin Hickey
2026-10-15 12:41:46 +00:00
committed by GitHub
parent c6b772cf6b
commit 252be28c18
9 changed files with 171 additions and 44 deletions
+6 -2
View File
@@ -154,11 +154,15 @@ should split those fields:
continue to work.
Callers that need only the token layout and item hashes, such as cache-aware
routers, can set `"return_mm_kwargs": false` on the render request. The response
routers, can set `"mm_payload": "none"` on the render request. The response
then keeps `mm_hashes` and `mm_placeholders` and sets `kwargs_data` and
`mm_metadata` to null, so the processed tensors are neither serialized nor sent.
Do not forward such a response to `/inference/v1/generate`, which reads a null
`kwargs_data` as every item being cached.
`kwargs_data` as every item being cached. The default, `"tensors"`, returns
both fields.
`"return_mm_kwargs": false` is a deprecated alias for `"mm_payload": "none"`
and is ignored when `mm_payload` is set.
## Example