[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
This commit is contained in:
@@ -7,16 +7,14 @@ Our renderer API is designed to disaggregate the render phase(preprocessing) and
|
||||
- Tokens-in / tokens-out engine: Make the engine a pure token-in / token-out service, decoupled from request preprocessing.
|
||||
|
||||
The dedicated `vllm launch render` server always exposes the `/render` and
|
||||
`/derender` endpoints when `VLLM_ENABLE_SCALE_OUT_ENDPOINTS` is unset or set to
|
||||
`1`. An explicit value of `0` conflicts with the renderer command and is
|
||||
rejected at startup.
|
||||
`/derender` endpoints.
|
||||
|
||||
Scale-out endpoints, including `/render`, `/derender`, and
|
||||
`/inference/v1/generate`, are disabled by default on a standard inference
|
||||
server. To expose them with `vllm serve`, opt in explicitly:
|
||||
|
||||
```bash
|
||||
VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 vllm serve <model>
|
||||
vllm serve <model> --enable-scale-out
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
Reference in New Issue
Block a user