[Bugfix][V1] Fix TOCTOU race causing intermittent EADDRINUSE on multi-API-server DP startup (#42585)

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
Vadim Gimpelson
2026-05-26 14:06:00 -07:00
committed by GitHub
co-authored by gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
parent d98cbf472b
commit 812e7e7364
7 changed files with 316 additions and 50 deletions
+14 -1
View File
@@ -308,7 +308,14 @@ def run_multi_api_server(args: argparse.Namespace):
from vllm.v1.engine.utils import get_engine_zmq_addresses
addresses = get_engine_zmq_addresses(vllm_config, num_api_servers)
# Per-API-server ports are picked by the kernel at each child's bind()
# to avoid parent-probe vs child-bind TOCTOU; Rust front-end opts out
# because it has no port-report-back channel.
addresses = get_engine_zmq_addresses(
vllm_config,
num_api_servers,
defer_api_server_ports=not rust_frontend_path,
)
with launch_core_engines(
vllm_config, executor_class, log_stats, addresses, num_api_servers
@@ -341,6 +348,12 @@ def run_multi_api_server(args: argparse.Namespace):
tensor_queue=tensor_queue,
)
# Forward each child's bound endpoints to the engine handshake
# (runs on ``with`` exit).
actual_inputs, actual_outputs = api_server_manager.gather_actual_addresses()
addresses.inputs = actual_inputs
addresses.outputs = actual_outputs
# Wait for API servers.
try:
wait_for_completion_or_failure(