[Docs] Remove references to removed env vars (#59530)
Signed-off-by: Sundri Lai <laiyanting.neu@gmail.com>
This commit is contained in:
@@ -83,10 +83,7 @@ directly from the Triton kernel through unified virtual addressing (UVA),
|
|||||||
prefetching on a side CUDA stream so transfers overlap with the compute of
|
prefetching on a side CUDA stream so transfers overlap with the compute of
|
||||||
preceding layers. The freed GPU memory goes to the KV cache.
|
preceding layers. The freed GPU memory goes to the KV cache.
|
||||||
|
|
||||||
`cpu_offload` defaults to `true` (on): it follows the
|
`cpu_offload` defaults to `true` (on)
|
||||||
`VLLM_PLE_CPU_OFFLOAD` environment variable, which also defaults to `true`.
|
|
||||||
An explicit `--engram-config` value takes precedence over the environment
|
|
||||||
variable.
|
|
||||||
|
|
||||||
To keep the tables resident on GPU instead (e.g. when host memory is
|
To keep the tables resident on GPU instead (e.g. when host memory is
|
||||||
scarce):
|
scarce):
|
||||||
@@ -96,8 +93,8 @@ vllm serve deepseek-ai/DeepSeek-V4.1-Flash \
|
|||||||
--engram-config.cpu_offload false
|
--engram-config.cpu_offload false
|
||||||
```
|
```
|
||||||
|
|
||||||
or equivalently `VLLM_PLE_CPU_OFFLOAD=0`. CPU offload requires a GPU with UVA
|
CPU offload requires a GPU with UVA support;
|
||||||
support; vLLM fails fast when it is unavailable.
|
vLLM fails fast when it is unavailable.
|
||||||
|
|
||||||
## Data-parallel topologies (DeepSeek V4.1)
|
## Data-parallel topologies (DeepSeek V4.1)
|
||||||
|
|
||||||
|
|||||||
@@ -51,8 +51,7 @@ and provision transport limits and memory accordingly.
|
|||||||
For example, start a standard inference server with scale-out endpoints enabled:
|
For example, start a standard inference server with scale-out endpoints enabled:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 \
|
vllm serve meta-llama/Llama-3.1-8B-Instruct --enable-scale-out
|
||||||
vllm serve meta-llama/Llama-3.1-8B-Instruct
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Send a Responses request and use `jq` to extract the `token_ids` field:
|
Send a Responses request and use `jq` to extract the `token_ids` field:
|
||||||
|
|||||||
Reference in New Issue
Block a user