[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
This commit is contained in:
@@ -13,9 +13,9 @@ zero client-side transformation.
|
||||
|
||||
Launch the server first:
|
||||
|
||||
VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 \
|
||||
vllm serve Qwen/Qwen3-VL-2B-Instruct \
|
||||
--dtype bfloat16 --max-model-len 4096 --enforce-eager
|
||||
vllm serve Qwen/Qwen3-VL-2B-Instruct \
|
||||
--dtype bfloat16 --max-model-len 4096 --enforce-eager \
|
||||
--enable-scale-out
|
||||
|
||||
Then run this script:
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
"""Call a token-in/token-out server started with:
|
||||
|
||||
VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 vllm serve Qwen/Qwen3-0.6B
|
||||
vllm serve Qwen/Qwen3-0.6B --enable-scale-out
|
||||
"""
|
||||
|
||||
import httpx
|
||||
|
||||
Reference in New Issue
Block a user