[Qwen4Exp] Pipeline-stage ownership for PLE, the MTP drafter and the vision tower
Each pipeline stage builds only the parts it owns: PLE when every PLE layer lives on stage 0, MTP under PP (port of vLLM PR #46994 minus its ROCm MLA file and tests) with the drafter on the last stage, and the vision tower on the first stage only. Co-Authored-By: Hinata <hinata@idihore.id> Signed-off-by: MOVZX <movzx@yahoo.com>
This commit is contained in:
@@ -883,12 +883,25 @@ class Qwen4ExpForConditionalGenerationConfig(Qwen3_5ForConditionalGenerationConf
|
||||
)
|
||||
# Checked again in Qwen4ExpModelState; rejecting it here keeps the
|
||||
# engine from loading weights first.
|
||||
# vllm#54709: PLE only needs raw input_ids on the rank that owns a PLE
|
||||
# layer. Allow PP>1 as long as every PLE layer lives on pipeline rank 0.
|
||||
if text_config.ple_layer_ids and parallel_config.pipeline_parallel_size > 1:
|
||||
raise NotImplementedError(
|
||||
"Qwen4Exp N-gram PLE embedding requires pipeline_parallel_size=1 "
|
||||
"because non-first pipeline ranks do not receive the raw input_ids "
|
||||
"it needs. Please run with PP=1."
|
||||
from vllm.distributed.utils import get_pp_indices
|
||||
|
||||
rank0_start, rank0_end = get_pp_indices(
|
||||
int(text_config.num_hidden_layers),
|
||||
0,
|
||||
parallel_config.pipeline_parallel_size,
|
||||
)
|
||||
ple_idx = [int(i) - 1 for i in text_config.ple_layer_ids]
|
||||
if min(ple_idx) < rank0_start or max(ple_idx) >= rank0_end:
|
||||
raise NotImplementedError(
|
||||
"Qwen4Exp N-gram PLE embedding requires every PLE layer to "
|
||||
f"live on pipeline rank 0 (layers {rank0_start}.."
|
||||
f"{rank0_end - 1}), but PLE layers are at {ple_idx}. Give "
|
||||
"rank 0 more layers via VLLM_PP_LAYER_PARTITION, or run "
|
||||
"with PP=1."
|
||||
)
|
||||
multimodal_config = vllm_config.model_config.multimodal_config
|
||||
if multimodal_config is not None and multimodal_config.language_model_only:
|
||||
_strip_qwen4_exp_mrope(vllm_config.model_config)
|
||||
|
||||
Reference in New Issue
Block a user