[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
62f3bf58a5
commit
3116c5d06b
@@ -47,7 +47,8 @@ class CudaCommunicator(DeviceCommunicatorBase):
|
||||
global_world_size,
|
||||
use_all2all=use_all2all,
|
||||
)
|
||||
if "tp" not in unique_name:
|
||||
# Match the group name exactly so ETP does not enable TP-only backends.
|
||||
if unique_name.split(":")[0] != "tp":
|
||||
# custom allreduce or torch symm mem can be used only by tp
|
||||
use_custom_allreduce = False
|
||||
use_torch_symm_mem = False
|
||||
|
||||
Reference in New Issue
Block a user