MOVZX
e0fe5df731
Merge branch 'main' into patches
2026-10-17 08:06:56 +00:00
Gabriel Wu and Claude Opus 5.5
dcd07967ca
[CPU] Keep the hybrid block size across the engine core's config rerun ( #61805 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 12:06:03 +08:00
djramic
307c5844ad
[ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test ( #62048 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-16 18:36:14 +00:00
MOVZX
be95edb7f6
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 17:35:42 +07:00
Stefano Castagnetta and Kevin H. Luu
784d1fff0c
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head ( #59973 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 22:06:40 +00:00
MOVZX
76df77c04e
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-15 21:16:39 +07:00
a400804ce4
[CI] Relax MTP sharded sampling cross-boot noise bound ( #57026 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 22:31:45 -07:00
Woosuk Kwon and Claude Opus 5.5
8530aed2b7
[Spec Decode] Remove Medusa and MLP speculator support ( #61701 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 17:29:23 -07:00
Taimys and OpenAI Codex
2c74fd1873
[Model Runner V2] Enable batch-sharded sampling for GPT-OSS ( #61257 )
...
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-14 10:11:11 -07:00
30e79c3a76
[Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) ( #52183 )
...
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com >
Signed-off-by: Andrey Talman <atalman@fb.com >
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 00:17:59 -07:00
Jiangyun Zhu and Claude Opus 5.5
badfcdbf91
[Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend ( #60050 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 14:49:46 +08:00
31e90ac370
[Attention] Enable FA4 sparse MQA decode on Blackwell ( #55866 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-13 09:31:37 -04:00
Jiangyun Zhu
ff305934f1
[Tests][Mamba] Wait for accepted counts in fake MTP proposer ( #61342 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-13 16:15:47 +08:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
Nick Hill
c059bee812
[MRV2] Switch CI tests still running on MRV1 unnecessarily ( #61014 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-11 09:57:07 -07:00
MOVZX
ac305c44cb
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-09 17:41:13 +00:00
Andrey Talman and Andrey Talman
89ac8d971b
[CI][Test] Allow 10% ngram acceptance drop under preemption on CUDA in test_async_scheduling_accuracy ( #60715 )
...
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com >
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com >
2026-10-09 16:59:17 +00:00
Jiangyun Zhu and Claude Opus 5.5
e265b77982
[CI][Mamba] Use well-conditioned prompts for FlashInfer ReplaySSM parity tests ( #60515 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 17:39:56 +08:00
Divakar Verma and Andreas Karatzas
efb8d85f24
[ROCm][CI] Add FP32 rounding slack to accuracy test near-tie upper bounds ( #60292 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-07 08:19:22 -05:00
MOVZX
85c31d47ad
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 22:06:10 +07:00
MOVZX
2877121b2d
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 21:59:03 +07:00
ca0df1f122
[Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing ( #59504 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-03 06:26:12 +00:00
Benedikt Falk
c4973d94fe
[Bugfix][Frontend] Avoid generation for empty streaming input ( #59015 )
...
Signed-off-by: Benedikt Falk <bfalk@nvidia.com >
2026-10-02 18:48:43 +00:00
Harry Mellor and Claude Opus 5.5
5d20979fc9
[Misc] Remove code paths for Transformers < 5.16.1 ( #59762 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:17:24 +00:00
af2a582d5c
[Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix ( #59293 )
...
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-02 11:21:20 +01:00
4056c8ac1f
[Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool ( #59309 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-01 23:04:21 -07:00
df8fd42116
[CI][Spec Decode] Add async scheduling accuracy tests ( #55840 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-01 01:06:24 +00:00
Andrii Skliar and Andrii Skliar
deb148e730
[Mamba] Add FlashInfer ReplaySSM support for MTP ( #52928 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-09-30 21:41:17 +00:00
Yueh-Ting (eop) Chen and Matthew Bonanni
ff1b87cca2
[Bugfix][HiSparse] Preserve host prefix publication after request completion ( #59007 )
...
Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-09-30 16:14:51 +00:00
90e13fc757
[Bugfix][HiSparse] Never allocate GPU pages without host backing ( #59036 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 04:13:29 +00:00
MOVZX
a63dca4afb
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 12:54:18 +07:00
Fanli Lin and Kunshang Ji
6a6c2952fa
[XPU][CI] Make test_mamba_prefix_cache block-size agnostic ( #58269 )
...
Signed-off-by: Lin, Fanli <fanli.lin@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-29 07:12:43 +08:00
MOVZX
76af434852
Merge remote-tracking branch 'upstream/main' into patches
2026-09-28 12:06:52 +07:00
924707f1bf
[CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs ( #57237 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 05:01:31 -07:00
MOVZX
5da84df6b7
[BugFix] Honor drop_eagle_block in MambaManager
...
On a hybrid model (full-attention + Mamba/GDN layers, e.g. Qwen3.6-35B-A3B),
enabling MTP/EAGLE speculative decoding together with prefix caching corrupts
the recurrent state, and the server serves wrong answers. #43559 reports
tool-eval accuracy dropping from ~90% to ~50%. Nothing raises an exception
and nothing shows up in the logs, so an operator has no signal that anything
is wrong, and prefix caching then spreads it, because the corrupted state is
hash-reachable and every later request sharing that prefix restores from it.
The root cause is the find_longest_cache_hit method on MambaManager,
at vllm/v1/core/single_type_kv_cache_manager.py:1049 on main, which takes
drop_eagle_block in its signature and never acts on it. Its body searches
right-to-left for the last cached block, builds [null]*i + [cached], and
breaks, referencing neither drop_eagle_block nor .pop(.
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-27 10:45:10 +07:00
927c87b347
[PCP][DCP] Support DCP target model with non-DCP Dspark ( #56723 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-26 18:12:35 -04:00
c0c35ddd8e
[Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need ( #58342 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 02:05:11 -07:00
Robert Shaw and Robert Shaw
c64b15cde5
[Scheduler] Soften Long Prefill Tokens Threshhold ( #57951 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-21 17:19:42 -07:00
0b7f11a1ee
[AuxOutput] Add block-keyed storage for routed-expert outputs ( #45635 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-21 10:25:05 +00:00
038cb14bd7
[CI][Bugfix] Correct the Laguna DFlash acceptance-length reference ( #57647 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-19 01:44:48 +00:00
Misha Goin
db7a24c230
[Perf] Add fused DFlash2 grouped convolution ( #55960 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-17 21:38:59 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Li, Jiang
b68408043d
[Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU ( #56985 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-16 15:49:17 +08:00
Nick Hill
7dbc3d62c3
[Cleanup] Remove vestigial tpu_input_batch.py ( #56898 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-14 21:29:15 -07:00
a5f6f61a8a
[CI][CPU] Add speculative-decoding coverage to CPU CI ( #54934 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
2026-09-14 19:51:38 +08:00
d43bb2f37f
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering ( #53781 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-12 00:36:47 -04:00
6fe67cbbf3
[Spec][V2] Support MTP speculative decoding under pipeline parallelism ( #46994 )
...
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-11 17:55:21 +00:00
Wentao Ye
5fe77aecfc
[Deprecation] Deprecate items scheduled for 0.29 ( #55353 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-11 08:17:02 -07:00