[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags (#61479)

Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Misha Goin
2026-10-15 01:08:02 -07:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 48f17b77bb
commit 1c4c319792
13 changed files with 118 additions and 36 deletions
+2 -2
View File
@@ -230,7 +230,7 @@ are internal constants (no env vars).
| Flag (`ParallelConfig` field) | Default | Meaning |
| --- | --- | --- |
| `--enable-shm-tensor-arena` / `--no-enable-shm-tensor-arena` (`enable_shm_tensor_arena`) | **off** | Opt in to route large CPU tensors through the arena (reserves slots in `/dev/shm`). Off = the out-of-band `_reduce_tensor` path only, identical to stock behavior. |
| `--enable-shm-tensor-arena` / `--disable-shm-tensor-arena` (`enable_shm_tensor_arena`) | **off** | Opt in to route large CPU tensors through the arena (reserves slots in `/dev/shm`). Off = the out-of-band `_reduce_tensor` path only, identical to stock behavior. |
Internal constants in `shm_tensor_arena.py`: **8 slots × 256 MB**; tensors larger
than a slot, or smaller than the **8 MB** divert threshold, take the out-of-band
@@ -268,7 +268,7 @@ than a slot, or smaller than the **8 MB** divert threshold, take the out-of-band
(arena + pinning); vision-encoder compute for the same image is ~80 ms.
4. **Outstanding — arena vs. out-of-band on `main`.** Re-run the same seeded A/B
with the baseline set to current `main` (i.e. `_reduce_tensor` on, arena off
via `--no-enable-shm-tensor-arena`) vs. arena on, to quantify the incremental N→1
via `--disable-shm-tensor-arena`) vs. arena on, to quantify the incremental N→1
copy + pinning gain at TP=4 (and ideally TP=2/8 to show the copy-count scaling).
## 5. Limitations and future work
+2 -2
View File
@@ -52,7 +52,7 @@ CUDA_VISIBLE_DEVICES=0 vllm serve Qwen/Qwen2.5-VL-3B-Instruct \
--port 19534 \
--gpu-memory-utilization 0.35 \
--enable-request-id-headers \
--no-enable-prefix-caching \
--disable-prefix-caching \
--max-num-batched-tokens 16384 \
--allowed-local-media-path /path/to/media \
--ec-transfer-config '{
@@ -72,7 +72,7 @@ CUDA_VISIBLE_DEVICES=1 vllm serve Qwen/Qwen2.5-VL-3B-Instruct \
--gpu-memory-utilization 0.75 \
--enable-mm-embeds \
--enable-request-id-headers \
--no-enable-prefix-caching \
--disable-prefix-caching \
--allowed-local-media-path /path/to/media \
--ec-transfer-config '{
"ec_connector": "ECMooncakeConnector",
+1 -1
View File
@@ -61,7 +61,7 @@ Normally the first serving engine pays for the pass, and on a large MoE model it
dominates what is left of the startup time once the weights come from the
daemon.
Unless FlashInfer autotune is disabled (`--no-enable-flashinfer-autotune`),
Unless FlashInfer autotune is disabled (`--disable-flashinfer-autotune`),
`vllm preload` pays it at preload time: each daemon runs the autotune pass in
place — profiling dummy runs against the shard it already holds — after
loading and before binding its socket, then writes the cache:
+1 -1
View File
@@ -355,4 +355,4 @@ vllm serve LiquidAI/d1-3B
- A `choice` whose option names are all single letters is labeled with those
letters.
- When each request carries one question over a state that changes from
request to request, `--no-enable-prefix-caching` lowers latency.
request to request, `--disable-prefix-caching` lowers latency.