[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm (#61471)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Andreas Karatzas
2026-10-15 20:11:17 -05:00
committed by GitHub
co-authored by Claude
parent a956a8126e
commit 29345f42a6
17 changed files with 361 additions and 83 deletions
+1 -1
View File
@@ -992,7 +992,7 @@ class Qwen4ExpForConditionalGenerationConfig(Qwen3_5ForConditionalGenerationConf
parallel_config = vllm_config.parallel_config
# MoE SP also enables HC SP without the explicit HC flag.
if (
current_platform.is_cuda()
current_platform.is_cuda_alike()
and parallel_config.use_sequence_parallel_moe
and parallel_config.pipeline_parallel_size != 1
):