[Frontend] Add stateless /v1/responses/render endpoint (#50195)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
co-authored by
OpenAI Codex
parent
5e6f6a8ed4
commit
6a2a2bb02b
@@ -141,6 +141,8 @@ For further details on renderer APIs, please refer to [this page](renderer.md).
|
||||
- Render completion requests
|
||||
- [Chat Completions Render API](renderer.md) (`/v1/chat/completions/render`)
|
||||
- Render chat completions
|
||||
- [Responses Render API](renderer.md) (`/v1/responses/render`)
|
||||
- Render self-contained Responses requests
|
||||
|
||||
### Derenderer APIs
|
||||
|
||||
|
||||
@@ -25,6 +25,29 @@ VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 vllm serve <model>
|
||||
- Render completion requests
|
||||
- [Chat Completions Render API](renderer.md) (`/v1/chat/completions/render`)
|
||||
- Render chat completions
|
||||
- [Responses Render API](renderer.md) (`/v1/responses/render`)
|
||||
- Render a self-contained Responses request
|
||||
|
||||
The Responses render endpoint uses the same prompt construction as
|
||||
`/v1/responses` and returns one token-in `GenerateRequest`. It is stateless:
|
||||
inline history is supported, but `previous_response_id` is not. Callers must
|
||||
resolve stored response state and include the resulting history in the request
|
||||
before rendering.
|
||||
|
||||
For multimodal requests, the `GenerateRequest` contains the model-processed
|
||||
multimodal payload, which can be substantially larger than the source image or
|
||||
video. The caller must forward that payload unchanged to the generation service
|
||||
and provision transport limits and memory accordingly.
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/v1/responses/render \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "meta-llama/Llama-3.1-8B-Instruct",
|
||||
"input": "Explain prefix caching in one sentence.",
|
||||
"max_output_tokens": 32
|
||||
}'
|
||||
```
|
||||
|
||||
For the post processing counterpart that turns generated token IDs back into OpenAI compatible responses, see the [Derenderer APIs](derenderer.md).
|
||||
|
||||
|
||||
@@ -170,6 +170,7 @@ When `--api-key` is configured, the following `/v1` endpoints require Bearer tok
|
||||
- `/v1/messages/render` - Render Anthropic-compatible messages (available on `vllm serve` only when `VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1`, or on `vllm launch render` unless explicitly disabled)
|
||||
- `/v1/messages/count_tokens` - Count tokens for Anthropic messages
|
||||
- `/v1/responses` - Create a response
|
||||
- `/v1/responses/render` - Render a self-contained response request (available on `vllm serve` only when `VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1`, or on `vllm launch render` unless explicitly disabled)
|
||||
- `/v1/responses/{response_id}` - Retrieve a response
|
||||
- `/v1/responses/{response_id}/cancel` - Cancel a response
|
||||
- `/v1/score` - Scoring API
|
||||
|
||||
Reference in New Issue
Block a user