243 Commits
Author SHA1 Message Date
MOVZX e0fe5df731 Merge branch 'main' into patches 2026-10-17 08:06:56 +00:00
Gabriel WuandClaude Opus 5.5 dcd07967ca [CPU] Keep the hybrid block size across the engine core's config rerun (#61805)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 12:06:03 +08:00
djramic 307c5844ad [ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test (#62048)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
2026-10-16 18:36:14 +00:00
MOVZX be95edb7f6 Merge branch 'main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-16 17:35:42 +07:00
Stefano CastagnettaandKevin H. Luu 784d1fff0c [MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head (#59973)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-15 22:06:40 +00:00
MOVZX 76df77c04e Merge branch 'main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-15 21:16:39 +07:00
a400804ce4 [CI] Relax MTP sharded sampling cross-boot noise bound (#57026)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 22:31:45 -07:00
Woosuk KwonandClaude Opus 5.5 8530aed2b7 [Spec Decode] Remove Medusa and MLP speculator support (#61701)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:29:23 -07:00
TaimysandOpenAI Codex 2c74fd1873 [Model Runner V2] Enable batch-sharded sampling for GPT-OSS (#61257)
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-14 10:11:11 -07:00
30e79c3a76 [Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) (#52183)
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Signed-off-by: Andrey Talman <atalman@fb.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 00:17:59 -07:00
Jiangyun ZhuandClaude Opus 5.5 badfcdbf91 [Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:49:46 +08:00
31e90ac370 [Attention] Enable FA4 sparse MQA decode on Blackwell (#55866)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-13 09:31:37 -04:00
Jiangyun Zhu ff305934f1 [Tests][Mamba] Wait for accepted counts in fake MTP proposer (#61342)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
2026-10-13 16:15:47 +08:00
Jiangyun ZhuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2cc2e68d1c [Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
Woosuk KwonandClaude Opus 5.5 e1a5086256 [Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 10:41:07 -07:00
Nick Hill c059bee812 [MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-10-11 09:57:07 -07:00
MOVZX ac305c44cb Merge branch 'main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-09 17:41:13 +00:00
Andrey TalmanandAndrey Talman 89ac8d971b [CI][Test] Allow 10% ngram acceptance drop under preemption on CUDA in test_async_scheduling_accuracy (#60715)
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
2026-10-09 16:59:17 +00:00
Jiangyun ZhuandClaude Opus 5.5 e265b77982 [CI][Mamba] Use well-conditioned prompts for FlashInfer ReplaySSM parity tests (#60515)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:39:56 +08:00
Divakar VermaandAndreas Karatzas efb8d85f24 [ROCm][CI] Add FP32 rounding slack to accuracy test near-tie upper bounds (#60292)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-10-07 08:19:22 -05:00
MOVZX 85c31d47ad Merge remote-tracking branch 'upstream/main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-03 22:06:10 +07:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-10-03 21:59:03 +07:00
ca0df1f122 [Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing (#59504)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-03 06:26:12 +00:00
Benedikt Falk c4973d94fe [Bugfix][Frontend] Avoid generation for empty streaming input (#59015)
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
2026-10-02 18:48:43 +00:00
Harry MellorandClaude Opus 5.5 5d20979fc9 [Misc] Remove code paths for Transformers < 5.16.1 (#59762)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 11:17:24 +00:00
af2a582d5c [Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix (#59293)
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-10-02 11:21:20 +01:00
4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-10-01 23:04:21 -07:00
df8fd42116 [CI][Spec Decode] Add async scheduling accuracy tests (#55840)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-01 01:06:24 +00:00
Andrii SkliarandAndrii Skliar deb148e730 [Mamba] Add FlashInfer ReplaySSM support for MTP (#52928)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
2026-09-30 21:41:17 +00:00
Yueh-Ting (eop) ChenandMatthew Bonanni ff1b87cca2 [Bugfix][HiSparse] Preserve host prefix publication after request completion (#59007)
Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
2026-09-30 16:14:51 +00:00
90e13fc757 [Bugfix][HiSparse] Never allocate GPU pages without host backing (#59036)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 04:13:29 +00:00
MOVZX a63dca4afb Merge remote-tracking branch 'upstream/main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 12:54:18 +07:00
Fanli LinandKunshang Ji 6a6c2952fa [XPU][CI] Make test_mamba_prefix_cache block-size agnostic (#58269)
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-29 07:12:43 +08:00
MOVZX 76af434852 Merge remote-tracking branch 'upstream/main' into patches 2026-09-28 12:06:52 +07:00
924707f1bf [CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs (#57237)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 05:01:31 -07:00
MOVZX 5da84df6b7 [BugFix] Honor drop_eagle_block in MambaManager
On a hybrid model (full-attention + Mamba/GDN layers, e.g. Qwen3.6-35B-A3B),
enabling MTP/EAGLE speculative decoding together with prefix caching corrupts
the recurrent state, and the server serves wrong answers. #43559 reports
tool-eval accuracy dropping from ~90% to ~50%. Nothing raises an exception
and nothing shows up in the logs, so an operator has no signal that anything
is wrong, and prefix caching then spreads it, because the corrupted state is
hash-reachable and every later request sharing that prefix restores from it.

The root cause is the find_longest_cache_hit method on MambaManager,
at vllm/v1/core/single_type_kv_cache_manager.py:1049 on main, which takes
drop_eagle_block in its signature and never acts on it. Its body searches
right-to-left for the last cached block, builds [null]*i + [cached], and
breaks, referencing neither drop_eagle_block nor .pop(.

Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-27 10:45:10 +07:00
927c87b347 [PCP][DCP] Support DCP target model with non-DCP Dspark (#56723)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-26 18:12:35 -04:00
c0c35ddd8e [Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-24 02:05:11 -07:00
Robert ShawandRobert Shaw c64b15cde5 [Scheduler] Soften Long Prefill Tokens Threshhold (#57951)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
2026-09-21 17:19:42 -07:00
0b7f11a1ee [AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-21 10:25:05 +00:00
038cb14bd7 [CI][Bugfix] Correct the Laguna DFlash acceptance-length reference (#57647)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-19 01:44:48 +00:00
Misha Goin db7a24c230 [Perf] Add fused DFlash2 grouped convolution (#55960)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-09-17 21:38:59 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
Li, Jiang b68408043d [Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
2026-09-16 15:49:17 +08:00
Nick Hill 7dbc3d62c3 [Cleanup] Remove vestigial tpu_input_batch.py (#56898)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-09-14 21:29:15 -07:00
a5f6f61a8a [CI][CPU] Add speculative-decoding coverage to CPU CI (#54934)
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
2026-09-14 19:51:38 +08:00
d43bb2f37f [3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
2026-09-12 00:36:47 -04:00
Chris EastwoodNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
6fe67cbbf3 [Spec][V2] Support MTP speculative decoding under pipeline parallelism (#46994)
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-11 17:55:21 +00:00
Wentao Ye 5fe77aecfc [Deprecation] Deprecate items scheduled for 0.29 (#55353)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-09-11 08:17:02 -07:00