iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

[Frontend] Add cache salt and tracing to structured decisions (#60786)

Browse Source 
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
This commit is contained in:
Taewoong Kim
2026-10-09 08:53:50 +00:00
committed by GitHub
co-authored by Chauncey
parent a6f43e57d6
commit 18c19eaf23
7 changed files with 207 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/serving/online_serving/structured_decisions.md
+5
View File
@@ -135,6 +135,11 @@ the request lists them, a `noul` labels its options `yes` and `no`, and a
| `questions` | question id to `{type, instructions, criteria}`, asked in this order |
| `instructions` | optional context placed ahead of the questions |
| `chat_template_kwargs` | passed to the chat template, for example `{"enable_thinking": false}` |
| `cache_salt` | optional [prefix-cache salt](../../usage/security.md#cache-salting) applied to every question |
A question with any field other than `type`, `instructions` and `criteria` is
rejected with a 400.
When [tracing](../../../examples/observability/opentelemetry/README.md) is enabled
with `--otlp-traces-endpoint`, the HTTP `traceparent` and `tracestate` headers are
forwarded to every question's generation request.