[Frontend] Gate scale-out endpoints behind opt-in flag (#54579)

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
This commit is contained in:
Francisco Javier Arceo
2026-09-02 14:31:14 +08:00
committed by GitHub
parent 300f688321
commit c00091e026
20 changed files with 386 additions and 39 deletions
+13
View File
@@ -6,6 +6,19 @@ Our renderer API is designed to disaggregate the render phase(preprocessing) and
- Disaggregated tokenization: Support use cases such as llm-d, Dynamo, and custom frontends that need to leverage vLLM's preprocessing logic without running the full inference engine.
- Tokens-in / tokens-out engine: Make the engine a pure token-in / token-out service, decoupled from request preprocessing.
The dedicated `vllm launch render` server always exposes the `/render` and
`/derender` endpoints when `VLLM_ENABLE_SCALE_OUT_ENDPOINTS` is unset or set to
`1`. An explicit value of `0` conflicts with the renderer command and is
rejected at startup.
Scale-out endpoints, including `/render`, `/derender`, and
`/inference/v1/generate`, are disabled by default on a standard inference
server. To expose them with `vllm serve`, opt in explicitly:
```bash
VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 vllm serve <model>
```
## API Reference
- [Completions Render API](renderer.md) (`/v1/completions/render`)