iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[CI] Pass the scale-out endpoint flag in EC E2E (#56819)

Browse Source 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
This commit is contained in:
Kevin H. Luu
2026-09-15 12:25:30 +08:00
committed by GitHub
co-authored by OpenAI Codex Tyler Michael Smith
parent a491b6039d
commit caf56ce9da
2 changed files with 5 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/usage/security.md
+1 -1
View File
@@ -170,7 +170,7 @@ When `--api-key` is configured, the following endpoints require Bearer token aut
- `/v1/messages/render` - Render Anthropic-compatible messages (available on `vllm serve` only when `--enable-scale-out` is set, or on `vllm launch render`)
- `/v1/messages/count_tokens` - Count tokens for Anthropic messages
- `/v1/responses` - Create a response
- `/v1/responses/render` - Render a self-contained response request (available on `vllm serve` only when `VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1`, or on `vllm launch render` unless explicitly disabled)
- `/v1/responses/render` - Render a self-contained response request (available on `vllm serve` only when `--enable-scale-out` is set, or on `vllm launch render` unless explicitly disabled)
- `/v1/responses/{response_id}` - Retrieve a response
- `/v1/responses/{response_id}/cancel` - Cancel a response
- `/v1/score` - Scoring API
tests/entrypoints/scale_out/ec_integration/run_scale_out_ec_e2e_test.sh
+4 -2
View File
@@ -181,8 +181,9 @@ run_scale_out_ec() {
# Encode-only EC producer instance. It runs the vision encoder and
# publishes embeddings through the ECExampleConnector shared storage.
echo "Starting encode instance on GPU $GPU_E, port $ENCODE_PORT"
env CUDA_VISIBLE_DEVICES="$GPU_E" VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 \
env CUDA_VISIBLE_DEVICES="$GPU_E" \
vllm serve "$MODEL" \
--enable-scale-out \
--port "$ENCODE_PORT" \
--max-model-len "$MAX_MODEL_LEN" \
--enforce-eager \
@@ -204,8 +205,9 @@ run_scale_out_ec() {
# features plus ec_transfer_params and loads the embeddings that the
# encode instance published.
echo "Starting prefill instance on GPU $GPU_PD, port $PREFILL_PORT"
env CUDA_VISIBLE_DEVICES="$GPU_PD" VLLM_ENABLE_SCALE_OUT_ENDPOINTS=1 \
env CUDA_VISIBLE_DEVICES="$GPU_PD" \
vllm serve "$MODEL" \
--enable-scale-out \
--port "$PREFILL_PORT" \
--max-model-len "$MAX_MODEL_LEN" \
--enforce-eager \