[Feature] Add --cpu-distributed-timeout-seconds CLI Option for CPU Process Group Timeout (#42968)
Signed-off-by: fangyuchu <fangyuchu@qq.com> Signed-off-by: zWaNg3 <389750525@qq.com> Co-authored-by: zWaNg3 <389750525@qq.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
zWaNg3
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
0f66623b0d
commit
e26e1f0928
@@ -292,6 +292,10 @@ class ParallelConfig:
|
||||
timeout parameter. If None, PyTorch's default timeout is used (600s for NCCL).
|
||||
Increase this for multi-node setups where model downloads may be slow."""
|
||||
|
||||
cpu_distributed_timeout_seconds: int | None = None
|
||||
"""Timeout (in seconds) for cpu communication groups. If None, PyTorch's
|
||||
default timeout is used (1800s for gloo)."""
|
||||
|
||||
world_size: int = Field(init=False)
|
||||
"""world_size is TPxPP, it affects the number of workers we create."""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user