[Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation (#57643)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> Co-authored-by: Codex <noreply@openai.com>
This commit is contained in:
@@ -512,6 +512,7 @@ def _check_dummy_hash_model_forward(
|
||||
model = SimpleNamespace(
|
||||
use_mega_moe=False,
|
||||
use_sequence_parallel=False,
|
||||
fuse_mhc_all_reduce=False,
|
||||
engram_hash=state,
|
||||
engram_swa_prefix="swa",
|
||||
engram_dp_shared_memory=dp_shared_memory,
|
||||
|
||||
Reference in New Issue
Block a user