[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479)
Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
48f17b77bb
commit
1c4c319792
@@ -230,7 +230,7 @@ are internal constants (no env vars).
|
||||
|
||||
| Flag (`ParallelConfig` field) | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `--enable-shm-tensor-arena` / `--no-enable-shm-tensor-arena` (`enable_shm_tensor_arena`) | **off** | Opt in to route large CPU tensors through the arena (reserves slots in `/dev/shm`). Off = the out-of-band `_reduce_tensor` path only, identical to stock behavior. |
|
||||
| `--enable-shm-tensor-arena` / `--disable-shm-tensor-arena` (`enable_shm_tensor_arena`) | **off** | Opt in to route large CPU tensors through the arena (reserves slots in `/dev/shm`). Off = the out-of-band `_reduce_tensor` path only, identical to stock behavior. |
|
||||
|
||||
Internal constants in `shm_tensor_arena.py`: **8 slots × 256 MB**; tensors larger
|
||||
than a slot, or smaller than the **8 MB** divert threshold, take the out-of-band
|
||||
@@ -268,7 +268,7 @@ than a slot, or smaller than the **8 MB** divert threshold, take the out-of-band
|
||||
(arena + pinning); vision-encoder compute for the same image is ~80 ms.
|
||||
4. **Outstanding — arena vs. out-of-band on `main`.** Re-run the same seeded A/B
|
||||
with the baseline set to current `main` (i.e. `_reduce_tensor` on, arena off
|
||||
via `--no-enable-shm-tensor-arena`) vs. arena on, to quantify the incremental N→1
|
||||
via `--disable-shm-tensor-arena`) vs. arena on, to quantify the incremental N→1
|
||||
copy + pinning gain at TP=4 (and ideally TP=2/8 to show the copy-count scaling).
|
||||
|
||||
## 5. Limitations and future work
|
||||
|
||||
@@ -52,7 +52,7 @@ CUDA_VISIBLE_DEVICES=0 vllm serve Qwen/Qwen2.5-VL-3B-Instruct \
|
||||
--port 19534 \
|
||||
--gpu-memory-utilization 0.35 \
|
||||
--enable-request-id-headers \
|
||||
--no-enable-prefix-caching \
|
||||
--disable-prefix-caching \
|
||||
--max-num-batched-tokens 16384 \
|
||||
--allowed-local-media-path /path/to/media \
|
||||
--ec-transfer-config '{
|
||||
@@ -72,7 +72,7 @@ CUDA_VISIBLE_DEVICES=1 vllm serve Qwen/Qwen2.5-VL-3B-Instruct \
|
||||
--gpu-memory-utilization 0.75 \
|
||||
--enable-mm-embeds \
|
||||
--enable-request-id-headers \
|
||||
--no-enable-prefix-caching \
|
||||
--disable-prefix-caching \
|
||||
--allowed-local-media-path /path/to/media \
|
||||
--ec-transfer-config '{
|
||||
"ec_connector": "ECMooncakeConnector",
|
||||
|
||||
@@ -61,7 +61,7 @@ Normally the first serving engine pays for the pass, and on a large MoE model it
|
||||
dominates what is left of the startup time once the weights come from the
|
||||
daemon.
|
||||
|
||||
Unless FlashInfer autotune is disabled (`--no-enable-flashinfer-autotune`),
|
||||
Unless FlashInfer autotune is disabled (`--disable-flashinfer-autotune`),
|
||||
`vllm preload` pays it at preload time: each daemon runs the autotune pass in
|
||||
place — profiling dummy runs against the shard it already holds — after
|
||||
loading and before binding its socket, then writes the cache:
|
||||
|
||||
@@ -355,4 +355,4 @@ vllm serve LiquidAI/d1-3B
|
||||
- A `choice` whose option names are all single letters is labeled with those
|
||||
letters.
|
||||
- When each request carries one question over a state that changes from
|
||||
request to request, `--no-enable-prefix-caching` lowers latency.
|
||||
request to request, `--disable-prefix-caching` lowers latency.
|
||||
|
||||
Reference in New Issue
Block a user