[Frontend] Gate scale-out endpoints behind opt-in flag (#54579)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
This commit is contained in:
@@ -121,6 +121,13 @@ For further details on profiling vLLM, please refer to [this page](../../contrib
|
||||
|
||||
## Scale-Out APIs
|
||||
|
||||
Scale-out APIs are disabled by default on `vllm serve`. The environment
|
||||
variable accepts only `0` or `1`; set `VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1` to
|
||||
register the endpoints below. The
|
||||
dedicated `vllm launch render` and `vllm serve --tokens-only` modes are explicit
|
||||
opt-ins and enable their required endpoints when the variable is unset; an
|
||||
explicit value of `0` is rejected for those modes.
|
||||
|
||||
### Tokens IN <> Tokens OUT APIs
|
||||
|
||||
- `/inference/v1/generate` - Generate completions
|
||||
|
||||
Reference in New Issue
Block a user