[Data-parallel] Allow DP>1 for world_size > num_gpus on node (8) (#26367)
Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com> Signed-off-by: Rui Qiao <ruisearch42@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
This commit is contained in:
co-authored by
gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Rui Qiao
parent
2ba60ec7fe
commit
b038d9c40c
@@ -1403,8 +1403,15 @@ class EngineArgs:
|
||||
"data_parallel_size_local must be set to use data_parallel_hybrid_lb."
|
||||
)
|
||||
|
||||
# Local DP size defaults to global DP size if not set.
|
||||
data_parallel_size_local = self.data_parallel_size
|
||||
if self.data_parallel_backend == "ray" and (
|
||||
envs.VLLM_RAY_DP_PACK_STRATEGY == "span"
|
||||
):
|
||||
# Data parallel size defaults to 1 if DP ranks are spanning
|
||||
# multiple nodes
|
||||
data_parallel_size_local = 1
|
||||
else:
|
||||
# Otherwise local DP size defaults to global DP size if not set
|
||||
data_parallel_size_local = self.data_parallel_size
|
||||
|
||||
# DP address, used in multi-node case for torch distributed group
|
||||
# and ZMQ sockets.
|
||||
|
||||
Reference in New Issue
Block a user