lesj0610
a66a3e1b4e
Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-14 14:36:05 +09:00
siyu and Isotr0py
52dd0d7562
[Fast Start] Fast loader support nnode>1 ( #55468 )
...
Signed-off-by: siyu <mg21330037@smail.nju.edu.cn >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-09-14 03:05:54 +00:00
Woosuk Kwon
3b533197a9
[Perf] Parallelize mHC pre-norm JIT warmup ( #56683 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-13 18:58:20 -07:00
lesj0610 and Claude Opus 5
3b1911c0ca
[Bugfix][Model] Fix Qwen4Exp MTP buffer placement, config parsing, and Triton warmup
...
Three initialization and config defects found serving Qwen4Exp, each with a
regression test.
**The MTP hidden buffer lands on the ambient device.** torch.empty for the
spec-decode hidden buffer takes no device, so it follows whatever device is
current rather than vllm_config.device_config.device. The allocation was
duplicated verbatim in the NVIDIA and AMD backends; it moves to
common/mtp.py so the two cannot drift, and the caller passes is_last_rank
rather than reaching for the process group from shared code.
**M-RoPE keys are reported as unrecognized.** Qwen4Exp drives M-RoPE through
mrope_section and mrope_interleaved, which transformers' rope validator does
not know for rope_type="default", so every load warns about a configuration
the model genuinely uses. Declare them at rope validation on the text config
and on the outer config that mirrors it; every other key still warns.
**transformers' layer_types spelling is unread.** Its Qwen4Exp config
canonicalizes "full_attention" entries to "qwen_sparse_attention" in memory,
so checkpoints re-exported through it serialize that name. vLLM keys QSA off
"full_attention" plus indexer_n_heads, so map the spelling back.
**Qwen4Exp is missing from the Triton warmup set.** Its GDN prefill and QSA
run on other backends, but the kernels this warmup covers -- the gated RMS
norm, the causal conv, the post-conv preparation, the decode state update --
are Triton whichever prefill kernel is chosen, so they compile on the first
prompt instead. qwen4_exp_mtp stays out: kernel_warmup passes the worker's
own model_config, which is the target model's.
tests/models/qwen4_exp/test_config.py: 15 passed.
tests/model_executor/test_qwen_triton_warmup.py: 6 passed.
ruff check, ruff format --check and mypy (Python 3.10) report what they
already report on the base commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com >
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-14 07:25:01 +09:00
Woosuk Kwon and OpenAI Codex
71888f507a
[Perf] Avoid redundant conversions in FP8 dummy initialization ( #56688 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-13 12:06:13 -07:00
Kevin H. Luu
dec0b5d63f
[Bugfix] Skip Triton autotune inspection without Triton ( #56676 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-09-13 17:05:29 +00:00
b7e0cdac5d
[Bugfix] Detect unloaded NVFP4 weight scales with a NaN sentinel ( #52501 )
...
Signed-off-by: pavelzak <pavel.zakharov@gmail.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-13 14:58:09 +00:00
Woosuk Kwon
e7a3963339
[Bugfix] Avoid repeated dummy initialization and random CPU Engram fills ( #56682 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-13 02:23:20 -07:00
Ylean and Codex
a987777755
[Bugfix][Gemma4] Keep image kwargs out of video preprocessing ( #56652 )
...
Signed-off-by: Ylean <61880605+Woolgathererer@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-13 09:07:12 +00:00
fa008bdccf
[Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer ( #56464 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-09-13 00:44:37 -07:00
Isotr0py
533dbe424b
[Misc] Clean up fast loader daemon quant method verification ( #55656 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-12 20:16:39 -07:00
Roberto L. Castro and Codex
e52be1a62d
[9/N][warmup][DSv4] Migrate MHC TileLang kernels ( #50178 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-12 17:25:23 -07:00
aed894c190
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels ( #56323 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-12 17:24:24 -07:00
ebe1dec2da
[PCP][DCP] Enable PCP+DCP on sparse-MLA models ( #56157 )
...
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai >
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-12 23:24:44 +00:00
7ee8a6dd01
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support ( #56122 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-09-12 18:42:01 +00:00
Ayushman Singh and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7bcca16729
[Bugfix] Fix DeepGEMM FP8 warmup coverage ( #56452 )
...
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-12 14:20:38 -04:00
Itay Alroy
658c8131c7
[Elastic EP] Reuse CUDA graphs across reconfiguration ( #54985 )
2026-09-12 10:26:39 -04:00
JartX
986e2f870a
[Refactor][ROCm] Migrate the RDNA3 W4A16 MoE to the oracle/experts pa… ( #55522 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-09-12 21:20:26 +08:00
Taneem Ibrahim
a0914ab7d0
[Nano-Nemotron] Fix Nano-Nemotron precomputed multimodal embeddings ( #56398 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-12 09:07:10 -04:00
d43bb2f37f
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering ( #53781 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-12 00:36:47 -04:00
Roberto L. Castro and Codex
120ec4ebd2
[5/N][warmup][DSv4] Migrate NVIDIA CuTeDSL attention kernels ( #53566 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-12 00:05:59 -04:00
Fangzhou Ai and Fangzhou Ai
46d2b23ac5
[ROCm][DSV4.1][Perf] Use AITER mHC for the delayed pre block ( #56503 )
...
Signed-off-by: Fangzhou Ai <fangzhou@semianalysis.ai >
Co-authored-by: Fangzhou Ai <fangzhou@semianalysis.ai >
2026-09-12 00:35:48 +00:00
Andy Lo
dc07f1638f
[Bugfix][MLA] Read sparse model settings from text config ( #56160 )
...
Signed-off-by: Andy Lo <andy@mistral.ai >
2026-09-11 18:02:13 +00:00
6fe67cbbf3
[Spec][V2] Support MTP speculative decoding under pipeline parallelism ( #46994 )
...
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-11 17:55:21 +00:00
Shanshan Shen
127143e27f
Revert "[Rocm][Kimi-k3] Fix pipeline_parallel support for the kimik3 DCP mode ( #53664 )" ( #56429 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-11 09:17:26 -07:00
Wentao Ye
5fe77aecfc
[Deprecation] Deprecate items scheduled for 0.29 ( #55353 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-11 08:17:02 -07:00
Harry Mellor and Claude Opus 5
c191787a68
[Bugfix] Align vLLM YaRN with Transformers and stop re-scaling max_model_len ( #56446 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 13:01:38 +00:00
labAxiaoming
a5b714eee4
[Bugfix] Fix Qwen3-VL and Cosmos3-Edge text architectures for CPU and pipeline parallelism ( #53699 )
...
Signed-off-by: xiaoming <1259730330@qq.com >
2026-09-11 05:58:15 -07:00
Yuxuan Zhang
89dbb26445
[Bugfix] Fix GLM-OCR MTP position masking during CUDA graph capture ( #56447 )
...
Signed-off-by: zRzRzRzRzRzRzR <Yuxuan.Zhang2@liverpool.ac.uk >
2026-09-11 12:31:10 +00:00
Matt
b4da4d17ae
[ROCm][Bugfix] Fix AITER preshuffled FP8 block-scale kernel ( #56433 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-09-11 11:44:57 +00:00
e77daef89e
[Model] Support DeepSeek-V4.1-Flash ( #56214 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 02:11:19 -07:00
Kevin H. Luu and OpenAI Codex
1cf6555214
[Bugfix] Pin EPLB and MLA host-to-device transfer buffers ( #56138 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-11 01:57:47 -07:00
fbf51c7026
[Bugfix] Fix FlashInfer KV sharing with omitted K/V ( #55864 )
...
Signed-off-by: zhicheng Liu <lzc021026@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 23:07:52 -07:00
980c16c8e4
[PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. ( #56107 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-10 22:07:53 -07:00
Marceli Fylcek and Claude Opus 5
5b6cf93e8e
[XPU] Add forward_xpu to Mixer2RMSNormGated and FusedRMSNormGated ( #54968 )
...
Signed-off-by: Marceli Fylcek <marceli.fylcek@intel.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-11 07:58:28 +08:00
Matt and Claude Opus 5
9163190dda
[ROCm][Bugfix][Perf] Tune multi-stream shared experts use; wvSplitKrc fixes ( #56098 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-10 15:14:22 -07:00
Micah Williamson
7de70fa7ae
[Bugfix][ROCm] Create linear layer biases with requires_grad=False ( #56161 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-09-10 13:54:52 -05:00
Simon Danielsson
7470082f57
[ROCm][Perf] Add bpreshuffled blockscaled fp8 GEMM ( #51692 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-10 16:18:37 +00:00
Itay Alroy and Tyler Michael Smith
bdea2777ea
[Bugfix][MoE] Fix batched CUTLASS workspace overallocation ( #55579 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-09-10 15:33:56 +00:00
Alexander Crettenand and Claude
7d8d71e989
[Bugfix] Qwen3-VL(-MoE): pass architectures to with_hf_config for pipeline parallelism ( #43272 )
...
Signed-off-by: Xonder <xonder@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-10 14:49:35 +01:00
Taneem Ibrahim
f9083cb83a
[Mypy] Fix typing for R/S models ( #54157 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-10 09:43:08 -04:00
siyu and Isotr0py
73fb19151f
[Fast Start] Support fp4 ( #55465 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 13:22:17 +00:00
YoavMiron and Benjamin Chislett
40e6042ec8
[Feature][Spec Decode] MTP with separate (possibly quantized) lm head for nemotron ( #54574 )
...
Signed-off-by: Yoav Miron <yomiron@nvidia.com >
Signed-off-by: YoavMiron <yomiron@nvidia.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-09-10 08:26:31 -04:00
9b959b8657
[Model] DeepSeek-V4.1-Flash Model Definitions ( #56228 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-10 20:16:26 +08:00
Jakub Byczkowski and Copilot
c9355e25e8
[XPU][Bugfix] Add forward_xpu to Ernie4_5_VLRotaryEmbedding ( #55942 )
...
Signed-off-by: Jakub Byczkowski <jakub.byczkowski@intel.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-09-10 17:48:00 +08:00
6fd08c45e6
[XPU][Bugfix] Fix FalconH1 pipeline-parallel execution ( #55913 )
...
Signed-off-by: Jakub Byczkowski <jakub.byczkowski@intel.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-10 17:47:29 +08:00
fe26c705f7
[Model] Add Bailing V3 VL support ( #55921 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Jared Wen and Claude Fable 5.1
1768273c13
[Perf][GLM-5.3-Flash] Decode hot-path cleanups: strided KDA recurrent inputs, NoPE MQA query without concat, no duplicate router GEMM ( #55736 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-10 16:28:33 +08:00
Yunzhu Lu and Isotr0py
442d36031c
[MM][CG] Enable encoder CUDA Graph for MiniCPM-V ( #42785 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 06:28:48 +00:00
Li, Jiang
c3ccc0e957
[Model] Add DeepSeek-V4 CPU backend ( #55355 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-10 14:14:48 +08:00
lesj0610