iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Frontend] Add cache salt and tracing to structured decisions (#60786)

Browse Source 
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
This commit is contained in:
Taewoong Kim
2026-10-09 08:53:50 +00:00
committed by GitHub
co-authored by Chauncey
parent a6f43e57d6
commit 18c19eaf23
7 changed files with 207 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/usage/security.md
+1 -1
View File
@@ -572,7 +572,7 @@ An attacker sharing the same backend can measure differences in Time to First To
vLLM accepts an optional `cache_salt` parameter on requests. The salt is mixed into the hash of the first KV cache block, so only requests carrying the same salt can share cached prefix blocks. See [Automatic Prefix Caching](../design/prefix_caching.md) for the implementation details.
`cache_salt` is accepted by the OpenAI-compatible chat completions, completions, responses, and pooling (embeddings, classification, scoring) endpoints, and by the Anthropic `/v1/messages` endpoint.
`cache_salt` is accepted by the OpenAI-compatible chat completions, completions, responses, and pooling (embeddings, classification, scoring) endpoints, the Anthropic `/v1/messages` endpoint, and the structured decisions `/v1/systemone` endpoint.
#### Usage with the OpenAI Python client