[Doc] Show how to get Responses prompt token IDs (#55912)

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Francisco Javier Arceo
2026-09-16 02:48:18 +00:00
committed by GitHub
co-authored by OpenAI Codex
parent b0898a4937
commit 7d4bba5498
3 changed files with 44 additions and 3 deletions
+6 -1
View File
@@ -130,7 +130,12 @@ Scale-out APIs are disabled by default on `vllm serve`. Set `--enable-scale-out`
### Renderer APIs
For further details on renderer APIs, please refer to [this page](renderer.md).
Renderer APIs preprocess completion, chat, and Responses requests without running
inference. They handle tokenization, model-specific prompt formatting, and
multimodal preprocessing, returning prompt token IDs, sampling parameters, and
any processed multimodal inputs for generation.
See the [renderer guide](renderer.md) for setup instructions and examples.
- [Completions Render API](renderer.md) (`/v1/completions/render`)
- Render completion requests
@@ -199,6 +199,11 @@ you can use the [official OpenAI Python client](https://github.com/openai/openai
Code example: [examples/tool_calling/openai_responses_client_with_tools.py](../../../examples/tool_calling/openai_responses_client_with_tools.py)
To get prompt token IDs before generation, use
[`/v1/responses/render`](renderer.md#get-responses-prompt-token-ids). It preprocesses
a self-contained Responses request without inference and returns the prompt
token IDs, so a separate `/tokenize` call is not needed.
#### Extra parameters
The following extra parameters in the request object are supported:
+33 -2
View File
@@ -26,27 +26,58 @@ vllm serve <model> --enable-scale-out
- [Responses Render API](renderer.md) (`/v1/responses/render`)
- Render a self-contained Responses request
## Get Responses prompt token IDs
Use `/v1/responses/render` to get prompt token IDs before choosing a model
replica. Rendering applies prompt construction and preprocessing without running
inference.
The Responses render endpoint uses the same prompt construction as
`/v1/responses` and returns one token-in `GenerateRequest`. It is stateless:
inline history is supported, but `previous_response_id` is not. Callers must
resolve stored response state and include the resulting history in the request
before rendering.
Configure the renderer and generation workers with the same model, tokenizer,
chat template, and preprocessing options. Render the full request, including
instructions, history, tools, and any template or truncation options, so the
returned IDs reflect the prompt the model will receive.
For multimodal requests, the `GenerateRequest` contains the model-processed
multimodal payload, which can be substantially larger than the source image or
video. The caller must forward that payload unchanged to the generation service
and provision transport limits and memory accordingly.
For example, start a standard inference server with scale-out endpoints enabled:
```bash
curl http://localhost:8000/v1/responses/render \
VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 \
vllm serve meta-llama/Llama-3.1-8B-Instruct
```
Send a Responses request and use `jq` to extract the `token_ids` field:
```bash
curl --fail --silent --show-error http://localhost:8000/v1/responses/render \
-H "Content-Type: application/json" \
-d '{
"model": "meta-llama/Llama-3.1-8B-Instruct",
"input": "Explain prefix caching in one sentence.",
"max_output_tokens": 32
}'
}' | jq '.token_ids'
```
To count the rendered prompt tokens for this text request, replace the `jq`
filter with `'.token_ids | length'`. The endpoint returns the full
`GenerateRequest`; `jq` filters the response on the client. Keep the complete
response if you will forward it to `/inference/v1/generate`, including any
multimodal features.
If the server has `--api-key` or `VLLM_API_KEY` configured, add
`-H "Authorization: Bearer <api-key>"` to the request. See
[API key authentication limitations](../../usage/security.md#api-key-authentication-limitations)
for the existing authentication boundaries.
For the post processing counterpart that turns generated token IDs back into OpenAI compatible responses, see the [Derenderer APIs](derenderer.md).
## Multimodal Render Features