[Qwen4Exp] Pipeline-stage ownership for PLE, the MTP drafter and the vision tower

Each pipeline stage builds only the parts it owns: PLE when every PLE layer
lives on stage 0, MTP under PP (port of vLLM PR #46994 minus its ROCm MLA file
and tests) with the drafter on the last stage, and the vision tower on the
first stage only.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
2026-10-09 19:43:53 +07:00
co-authored by Hinata
parent 461a8a0f82
commit 7e5f658d75
5 changed files with 170 additions and 18 deletions
+17 -4
View File
@@ -883,12 +883,25 @@ class Qwen4ExpForConditionalGenerationConfig(Qwen3_5ForConditionalGenerationConf
)
# Checked again in Qwen4ExpModelState; rejecting it here keeps the
# engine from loading weights first.
# vllm#54709: PLE only needs raw input_ids on the rank that owns a PLE
# layer. Allow PP>1 as long as every PLE layer lives on pipeline rank 0.
if text_config.ple_layer_ids and parallel_config.pipeline_parallel_size > 1:
raise NotImplementedError(
"Qwen4Exp N-gram PLE embedding requires pipeline_parallel_size=1 "
"because non-first pipeline ranks do not receive the raw input_ids "
"it needs. Please run with PP=1."
from vllm.distributed.utils import get_pp_indices
rank0_start, rank0_end = get_pp_indices(
int(text_config.num_hidden_layers),
0,
parallel_config.pipeline_parallel_size,
)
ple_idx = [int(i) - 1 for i in text_config.ple_layer_ids]
if min(ple_idx) < rank0_start or max(ple_idx) >= rank0_end:
raise NotImplementedError(
"Qwen4Exp N-gram PLE embedding requires every PLE layer to "
f"live on pipeline rank 0 (layers {rank0_start}.."
f"{rank0_end - 1}), but PLE layers are at {ple_idx}. Give "
"rank 0 more layers via VLLM_PP_LAYER_PARTITION, or run "
"with PP=1."
)
multimodal_config = vllm_config.model_config.multimodal_config
if multimodal_config is not None and multimodal_config.language_model_only:
_strip_qwen4_exp_mrope(vllm_config.model_config)