MOVZX and Hinata
ae0350c0de
[Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863 )
...
Intake of open upstream PR vllm-project/vllm#58863 . Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.
Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.
RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.
Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:39:09 +07:00
MOVZX and Hinata
a1d3c2cf1e
[Spec Decode] Fused draft metadata updates for the QSA side cache (PR #58449 )
...
Intake of open upstream PR vllm-project/vllm#58449 .
With MTP num_speculative_tokens > 1, the V2 autoregressive speculator
reuses one attention-metadata build across draft steps only when every
group's builder sets supports_draft_decode_metadata_update. The QSA
side-cache builder (QWEN4_EXP_EXP_QSA_STATE) did not, so every draft
step rebuilt attention metadata for the whole model.
QSAMetadataBuilder now implements update_draft_decode_metadata: it
re-launches the kernel with in-place seq_lens updates, so one metadata
build serves every draft step.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:36:09 +07:00
MOVZX and Hinata
cb275fdf20
[Qwen4Exp] Fix MTP buffer placement, config parsing, Triton warmup (PR #56742 )
...
Intake of open upstream PR vllm-project/vllm#56742 . Every rig here runs
MTP on both Qwen3.8 models, so these three bugs sit on our hot path.
The MTP hidden buffer was allocated with torch.empty and no device, so
it followed the ambient default instead of device_config.device; on a
CPU-default process the draft buffer lands on the wrong device. Both
NVIDIA and AMD backends now call make_mtp_hidden_buffer() in
vllm/models/qwen4_exp/common/mtp.py, which pins it to the configured
device.
Config parsing now accepts configs carrying M-RoPE keys and the
Transformers spelling of layer_types; both previously failed at load.
Triton warmup: Qwen4Exp was missing from the Qwen warmup dispatch, so
its Triton kernels compiled during the first served requests instead of
at startup.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:36:05 +07:00
MOVZX
a63dca4afb
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 12:54:18 +07:00
Bugen Zhao and Claude Opus 5.5
28f6739576
[Rust Frontend] Relax schema-aware tool argument conversion ( #59004 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 05:40:41 +00:00
Ren Yuzhou and Claude
ec5e0c352f
[Bugfix][Engram] Keep THP tables private when resolving shared memory ( #59068 )
...
Signed-off-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-29 05:19:49 +00:00
Shinichi Hemmi
17cc3cd80e
[Model] Engine based plamo3 parser ( #56575 )
...
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp >
2026-09-29 05:17:42 +00:00
shaohuaxi and Codex
36768d1bfd
[Bugfix][Frontend] Preserve caller parameters in offline pooling ( #58842 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 05:02:59 +00:00
Bowen Wang and Claude Opus 5.5
1fde8ff7da
[Bugfix] Detect the CUDA toolkit the way FlashInfer does in has_flashinfer() ( #58360 )
...
Signed-off-by: Bowen Wang <bowen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 04:14:49 +00:00
debanshd
13c8589b86
fix: return loaded parameters in Aria load_weights ( #57476 )
...
Signed-off-by: Debanshu Das <debanshu@google.com >
2026-09-29 04:05:07 +00:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00
cjackal and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
491f44adfa
[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next ( #51289 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 11:44:54 +08:00
Isotr0py and cjackal
07f1ba2ec6
[Multimodal] Remove processsor fallback for fused input norm params resolve ( #59073 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Isotr0py <2037008807@qq.com >
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-09-29 03:33:43 +00:00
Kevin H. Luu and Claude Opus 5.5
aedaba8664
[CI/Build] Skip the snapshot runtime on CUDA 12.x images ( #59118 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 20:17:59 -07:00
4e59d53ab5
[MRV2] Support stock torch.compile mode ( #59079 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-28 20:17:48 -07:00
Tyler Michael Smith and Codex
4406874e7c
[Bugfix][FT] Pass stateless process-group timeouts explicitly ( #58611 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 03:14:02 +00:00
Fangzhou Ai and Cursor
cf971b419a
[Bugfix] Revert "[ROCm][Perf] Replace torch.topk in DSA candidate block selection" ( #59125 )
...
Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-28 20:11:48 -07:00
c6843d397c
[Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo ( #55771 )
...
Signed-off-by: Shijie Lyu <sjlyu@amazon.com >
Co-authored-by: Shijie Lyu <sjlyu@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-29 10:36:22 +08:00
Muhammad Ahmed
0af34418e9
[ROCm][DSv4.1][Perf] Run the delayed mHC seams through aiter's fused Triton kernel ( #58655 )
...
Signed-off-by: Ahmed <Muhammad.Ahmed@amd.com >
2026-09-29 10:19:10 +08:00
Chaojun Zhang
7230dfea50
[XPU] Preserve non-contiguous strides when pinning CPU tensors for UVA view ( #54874 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-09-29 09:43:07 +08:00
Mehmet Cagri
70ae0b7435
[ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter ( #53492 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Signed-off-by: Mehmet Cagri <mehmet.kaymak@amd.com >
2026-09-29 09:08:18 +08:00
Andrey Talman
869278cbb0
[CI] Bound the CRCR report by build age, and stop gating it on a job ( #59066 )
2026-09-28 17:42:32 -07:00
Laith Sakka
b39f760cc7
[torch.compile] Canonicalize functionalized split slices for fusion pa… ( #57299 )
...
Signed-off-by: Laith Sakka <lsakka@meta.com >
2026-09-28 20:21:59 -04:00
Almog Tavor
171d1b9ef6
[Elastic EP] Support Model Runner V2 ( #53934 )
...
Signed-off-by: almogtavor <almogtavor@gmail.com >
Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com >
2026-09-29 00:20:40 +00:00
Benedikt Falk
c68eb98b69
[Bugfix][Frontend] Keep in-flight requests on the same DP engine ( #59017 )
...
Signed-off-by: Benedikt Falk <bfalk@nvidia.com >
2026-09-28 23:52:29 +00:00
Mehmet Cagri and Claude Opus 5.5
75fad5bbef
[Bugfix][ROCm] Drop -1 sentinels when building the ragged sparse-MLA indices ( #58058 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 18:40:11 -05:00
9b29370524
[Bugfix] Fix moe_wna16 w13 zero-point shard split for 8-bit asym GPTQ MoE ( #58950 )
...
Signed-off-by: Spandan Tiwari <sptiwari@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-28 23:38:28 +00:00
8a497d7557
[CI/Build][BugFix][The Rock] Make supports_mm_prefix return False for ROCm attn and unified attn since Prefix-LM not implemented ( #52395 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-28 23:30:38 +00:00
e03875f4d2
[Minimax M3] Enable fp8 indexer cache on triton indexer for non-SM100 architectures. ( #59081 )
...
Signed-off-by: Jarrel Seah <jarrel@harrison.ai >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Signed-off-by: Rajas Mhaskar <rmhaskar@nvidia.com >
Co-authored-by: Jarrel Seah <jarrel@harrison.ai >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 19:21:13 -04:00
Fanli Lin and Kunshang Ji
6a6c2952fa
[XPU][CI] Make test_mamba_prefix_cache block-size agnostic ( #58269 )
...
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-29 07:12:43 +08:00
Yan Ma
3bd7302396
[XPU] skip test_online_quantization_loads_real_weights ( #58987 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-09-29 07:10:52 +08:00
Yan Ma and Kunshang Ji
754aa29c00
[XPU] use rms_norm xpu kernel for context-key normalization ( #58896 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-29 07:09:58 +08:00
Abhinandan
32cc3f1ea8
[CI] Drop test-group rules that no longer match the tree ( #58895 )
...
Signed-off-by: Abhinandan <abhinandanverma551@gmail.com >
2026-09-28 21:46:21 +00:00
Giancarlo Delfin and Woosuk Kwon
3d5f4d4cd5
[Perf][Spec Decode] Avoid triton recompiles in the acceptance estimator ( #57107 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-09-28 13:46:25 -07:00
0xsensei and OpenAI Codex
d28795f1a7
[Bugfix][Scheduler] Refresh max tokens for streaming continuations ( #57676 )
...
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 20:31:57 +00:00
beef814e1d
[ROCm][CI] Add missing test coverage for upstream parity ( #50519 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-28 13:12:07 -07:00
fedbc3b564
[Bugfix][MRV2][Spec Decode] Reject draft slots that were never proposed ( #58784 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-28 13:07:15 -07:00
Nick Hill and Robert Shaw
42dbe4578c
[Core] Rework scheduler skipped_waiting queue ( #58947 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-09-28 20:03:16 +00:00
Ashraf Bhuiyan and Wentao Ye
2619b6b72b
[Mypy] Fix mypy typing for Voxtral and vision models ( #58251 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-28 15:52:50 -04:00
0xsensei
2840ca7a33
[Bugfix][Scheduler] Preserve logprobs across streaming continuations ( #57447 )
...
Signed-off-by: 0xsensei <prblmslvr.aditya@gmail.com >
2026-09-28 19:50:27 +00:00
e7c903609f
[Perf][MRV2] Allow FULL decode graphs for one-token prompt tails ( #58400 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-28 12:48:52 -07:00
Xiaoshuang Wang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ef42093a36
[Model Runner V2][Spec Decode] Support spec decode with draft model ( #43091 )
...
Signed-off-by: Icey <1790571317@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 19:38:05 +00:00
yzong-rh
2f4a975901
[Bugfix] Fix resumable request + async scheduling handoff race ( #58259 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-09-28 15:26:57 -04:00
764413559a
[Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode ( #58498 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-28 12:14:21 -07:00
25624f65cf
[ROCm][MLA] Add an AITER ASM round-robin decode route for DCP multi-token verify ( #56861 )
...
Signed-off-by: Xiaohu Guo <Xiaohu.Guo@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-28 14:52:21 -04:00
djramic and Andreas Karatzas
5ba2e3090b
[ROCm][CI] Increase timeout for Entrypoints Unit ( #59076 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-28 17:56:10 +00:00
Rohan Potdar and Claude Opus 5.5
7a160c6fe5
[ROCm][CI] Pin OpenTelemetry to LMCache's cap in the ROCm images ( #59051 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 12:33:09 -05:00
stefankoncarevic and Andreas Karatzas
6c300ddb9e
[CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP ( #59008 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-28 17:29:59 +00:00
c01368fd64
[Quant] Use canonical N-first weight format for CT WNA16 MoE ( #52798 )
...
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user ","status":"404"}>
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-28 11:22:44 -06:00
qizixi
6d3ea3c2c9
[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill ( #50499 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
2026-09-28 18:50:46 +02:00