[Frontend] Add cache salt support to generative scoring (#60841)

Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
This commit is contained in:
Taewoong Kim
2026-10-09 16:07:50 +00:00
committed by GitHub
co-authored by Cyrus Leung
parent 90ba2f34b5
commit 75e3b17c05
5 changed files with 68 additions and 9 deletions
@@ -62,3 +62,10 @@ Here, each item is appended to the query to form prompts like `"Is this city the
"usage": {"prompt_tokens": 45, "total_tokens": 48, "completion_tokens": 3}
}
```
## Cache salting
The optional `cache_salt` request field is applied to every item's prompt. With
prefix caching enabled, requests with different salts do not share cached prefix
blocks. See [Cache Salting](../../usage/security.md#cache-salting) for how to choose
and manage salts.
+1 -1
View File
@@ -572,7 +572,7 @@ An attacker sharing the same backend can measure differences in Time to First To
vLLM accepts an optional `cache_salt` parameter on requests. The salt is mixed into the hash of the first KV cache block, so only requests carrying the same salt can share cached prefix blocks. See [Automatic Prefix Caching](../design/prefix_caching.md) for the implementation details.
`cache_salt` is accepted by the OpenAI-compatible chat completions, completions, responses, and pooling (embeddings, classification, scoring) endpoints, the Anthropic `/v1/messages` endpoint, and the structured decisions `/v1/systemone` endpoint.
`cache_salt` is accepted by the OpenAI-compatible chat completions, completions, responses, and pooling (embeddings, classification, scoring) endpoints, the Anthropic `/v1/messages` endpoint, the structured decisions `/v1/systemone` endpoint, and the generative scoring `/generative_scoring` endpoint.
#### Usage with the OpenAI Python client