[Bug] Fix status update address for non-MOE model within external dp mode (#40839)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
This commit is contained in:
Wentao Ye
2026-05-04 16:37:16 -07:00
committed by GitHub
parent 1cb0838721
commit 577b9623e6
5 changed files with 19 additions and 7 deletions
+1 -1
View File
@@ -98,7 +98,7 @@ For larger scale deployments especially, it can make sense to handle the orchest
In this case, it's more convenient to treat each DP rank like a separate vLLM deployment, with its own endpoint, and have an external router balance HTTP requests between them, making use of appropriate real-time telemetry from each server for routing decisions.
This can already be done trivially for non-MoE models, since each deployed server is fully independent. No data parallel CLI options need to be used for this.
This can already be done trivially for non-MoE models, since each deployed server is fully independent. In that case, launch independent vLLM instances without any `--data-parallel-*` arguments; external DP CLI options are only supported for MoE deployments.
We support an equivalent topology for MoE DP+EP which can be configured via the following CLI arguments.