[Bugfix][Model] Fix Qwen4Exp MTP buffer placement, config parsing, and Triton warmup
Three initialization and config defects found serving Qwen4Exp, each with a regression test. **The MTP hidden buffer lands on the ambient device.** torch.empty for the spec-decode hidden buffer takes no device, so it follows whatever device is current rather than vllm_config.device_config.device. The allocation was duplicated verbatim in the NVIDIA and AMD backends; it moves to common/mtp.py so the two cannot drift, and the caller passes is_last_rank rather than reaching for the process group from shared code. **M-RoPE keys are reported as unrecognized.** Qwen4Exp drives M-RoPE through mrope_section and mrope_interleaved, which transformers' rope validator does not know for rope_type="default", so every load warns about a configuration the model genuinely uses. Declare them at rope validation on the text config and on the outer config that mirrors it; every other key still warns. **transformers' layer_types spelling is unread.** Its Qwen4Exp config canonicalizes "full_attention" entries to "qwen_sparse_attention" in memory, so checkpoints re-exported through it serialize that name. vLLM keys QSA off "full_attention" plus indexer_n_heads, so map the spelling back. **Qwen4Exp is missing from the Triton warmup set.** Its GDN prefill and QSA run on other backends, but the kernels this warmup covers -- the gated RMS norm, the causal conv, the post-conv preparation, the decode state update -- are Triton whichever prefill kernel is chosen, so they compile on the first prompt instead. qwen4_exp_mtp stays out: kernel_warmup passes the worker's own model_config, which is the target model's. tests/models/qwen4_exp/test_config.py: 15 passed. tests/model_executor/test_qwen_triton_warmup.py: 6 passed. ruff check, ruff format --check and mypy (Python 3.10) report what they already report on the base commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
319cc5ef19
commit
3b1911c0ca
@@ -22,6 +22,13 @@ _QWEN_MODEL_TYPES = frozenset(
|
||||
"qwen3_5_text",
|
||||
"qwen3_5_moe",
|
||||
"qwen3_5_moe_text",
|
||||
# Qwen4Exp runs its GDN prefill and QSA on other backends, but the
|
||||
# kernels this file warms are the Triton ones around them -- the gated
|
||||
# RMS norm, the causal conv, the post-conv preparation, the decode
|
||||
# state update -- and those stay whichever prefill kernel is chosen.
|
||||
# Left out of this set they compile on the first prompt instead.
|
||||
"qwen4_exp",
|
||||
"qwen4_exp_text",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user