[Render] Add round trip parity test and docs for derender (#48617)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
This commit is contained in:
Martin Hickey
2026-07-17 05:02:46 +00:00
committed by GitHub
parent 67fe73b2b4
commit 4d4e04f452
7 changed files with 397 additions and 2 deletions
+6 -2
View File
@@ -137,8 +137,12 @@ For further details on renderer APIs, please refer to [this page](renderer.md).
### Derenderer APIs
- `/v1/completions/derender` - Derenderer completion requests
- `/v1/chat/completions/derender` - Derenderer chat completion requests
For further details on derenderer APIs, please refer to [this page](derenderer.md).
- [Chat Completions Derender API](derenderer.md) (`/v1/chat/completions/derender`)
- Derender chat completion requests
- [Completions Derender API](derenderer.md) (`/v1/completions/derender`)
- Derender completion requests
## Tokenize APIs
+98
View File
@@ -0,0 +1,98 @@
# Derenderer APIs
The derenderer API is the post processing counterpart to the [Renderer APIs](renderer.md). Where `/render` turns a request into token ID (preprocessing), `/derender` turns generated token IDs back into a fully formed OpenAI compatible response (detokenization, reasoning parsing, tool call parsing), all without a GPU.
This closes the loop for a token-in / token-out engine in disaggregated serving:
- **GPU less post processing**: Detokenization, reasoning parsing, and tool call parsing run on the same GPU less frontend that hosts `/render`
- **Parser parity**: The derenderer reuses vLLM's tool and reasoning parsers, so a disaggregated deployment produces the same `content`/`reasoning`/ `tool_calls` split as a standard `vllm serve` server
- **Non-streaming**: The endpoints expect a complete `GenerateResponse` with all token IDs present and perform one-shot parsing. Streaming derender would require a separate endpoint design and is not currently supported but is in the pipeline
Both endpoints are hosted by the GPU less rendering server started with [`vllm launch render`](../../cli/launch/render.md), alongside the `/render`
endpoints.
## Pipeline
```text
render generate derender
request ───────────────▶ token_ids ─────────▶ token_ids ──────────▶ response
(chat / (GPU less) (token-in / (GPU less) (OpenAI
completion) │ token-out engine) ▲ compatible)
└─────────────── request + prompt_tokens ──┘
```
The derender step needs more than the engine's `token_ids`. It also consumes the original `chat_request`/`completion_request` and `prompt_tokens` carried over from the render step (see [Request format](#request-format)) so the tool and reasoning parsers have the context they need.
## API Reference
- Chat Completions Derender API (`/v1/chat/completions/derender`)
- Post process a single `GenerateResponse` into a `ChatCompletionResponse`
- Completions Derender API (`/v1/completions/derender`)
- Post process a list of `GenerateResponse` objects (one per prompt) into a `CompletionResponse`
## Request format
Each request wraps the engine's `GenerateResponse`(s) together with the caller metadata needed to reconstruct the final response without a GPU.
`/v1/chat/completions/derender`:
??? code
```python
--8<-- "vllm/entrypoints/scale_out/token_in_token_out/protocol.py:derender-chat-request"
```
`/v1/completions/derender`:
??? code
```python
--8<-- "vllm/entrypoints/scale_out/token_in_token_out/protocol.py:derender-completion-request"
```
Oversized payloads are rejected with a `400` before any `tokenizer.decode()` or parser runs.
## Example
The example below drives the full `render → generate → derender` round trip for a chat request against a GPU less render server (`/render`, `/derender`) and a token-in / token-out engine (`/inference/v1/generate`).
```python
import httpx
MODEL = "meta-llama/Llama-3.2-1B-Instruct"
RENDER = "http://localhost:8100" # vllm launch render ...
ENGINE = "http://localhost:8200" # token-in / token-out engine
chat_request = {
"model": MODEL,
"messages": [{"role": "user", "content": "What is 2+2?"}],
"max_tokens": 32,
}
with httpx.Client(timeout=60.0) as client:
# 1. Render: request -> token IDs (GPU less)
generate_request = client.post(
f"{RENDER}/v1/chat/completions/render", json=chat_request
).json()
prompt_tokens = len(generate_request["token_ids"])
# 2. Generate: token IDs -> token IDs (token-in / token-out engine)
generate_response = client.post(
f"{ENGINE}/inference/v1/generate", json=generate_request
).json()
# 3. Derender: token IDs -> ChatCompletionResponse (GPU less)
response = client.post(
f"{RENDER}/v1/chat/completions/derender",
json={
"model": MODEL,
"generate_response": generate_response,
"prompt_tokens": prompt_tokens,
"chat_request": chat_request,
},
).json()
print(response["choices"][0]["message"]["content"])
```
Passing `chat_request` lets the derenderer run the configured tool and reasoning parsers. This means `response["choices"][0]["message"]` carries the same `content` / `reasoning` / `tool_calls` split a `vllm serve` server would produce. Omit `chat_request` for plain detokenization only.
+2
View File
@@ -12,3 +12,5 @@ Our renderer API is designed to disaggregate the render phase(preprocessing) and
- Render completion requests
- [Chat Completions Render API](renderer.md) (`/v1/chat/completions/render`)
- Render chat completions
For the post processing counterpart that turns generated token IDs back into OpenAI compatible responses, see the [Derenderer APIs](derenderer.md).