|
31e90ac370
|
[Attention] Enable FA4 sparse MQA decode on Blackwell (#55866)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-10-13 09:31:37 -04:00 |
|
|
Jiangyun Zhu
|
ff305934f1
|
[Tests][Mamba] Wait for accepted counts in fake MTP proposer (#61342)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-10-13 16:15:47 +08:00 |
|
|
|
2cc2e68d1c
|
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 00:58:59 -07:00 |
|
|
Woosuk KwonandClaude Opus 5.5
|
e1a5086256
|
[Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 10:41:07 -07:00 |
|
|
Nick Hill
|
c059bee812
|
[MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-11 09:57:07 -07:00 |
|
|
Andrey TalmanandAndrey Talman
|
89ac8d971b
|
[CI][Test] Allow 10% ngram acceptance drop under preemption on CUDA in test_async_scheduling_accuracy (#60715)
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
|
2026-10-09 16:59:17 +00:00 |
|
|
Jiangyun ZhuandClaude Opus 5.5
|
e265b77982
|
[CI][Mamba] Use well-conditioned prompts for FlashInfer ReplaySSM parity tests (#60515)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 17:39:56 +08:00 |
|
|
Divakar VermaandAndreas Karatzas
|
efb8d85f24
|
[ROCm][CI] Add FP32 rounding slack to accuracy test near-tie upper bounds (#60292)
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-07 08:19:22 -05:00 |
|
|
|
ca0df1f122
|
[Bugfix][Core] Exempt exactly the blocks an async KV load writes from zeroing (#59504)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-03 06:26:12 +00:00 |
|
|
Benedikt Falk
|
c4973d94fe
|
[Bugfix][Frontend] Avoid generation for empty streaming input (#59015)
Signed-off-by: Benedikt Falk <bfalk@nvidia.com>
|
2026-10-02 18:48:43 +00:00 |
|
|
Harry MellorandClaude Opus 5.5
|
5d20979fc9
|
[Misc] Remove code paths for Transformers < 5.16.1 (#59762)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-02 11:17:24 +00:00 |
|
|
|
af2a582d5c
|
[Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix (#59293)
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-02 11:21:20 +01:00 |
|
|
|
4056c8ac1f
|
[Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-10-01 23:04:21 -07:00 |
|
|
|
df8fd42116
|
[CI][Spec Decode] Add async scheduling accuracy tests (#55840)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-01 01:06:24 +00:00 |
|
|
Andrii SkliarandAndrii Skliar
|
deb148e730
|
[Mamba] Add FlashInfer ReplaySSM support for MTP (#52928)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-09-30 21:41:17 +00:00 |
|
|
Yueh-Ting (eop) ChenandMatthew Bonanni
|
ff1b87cca2
|
[Bugfix][HiSparse] Preserve host prefix publication after request completion (#59007)
Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-09-30 16:14:51 +00:00 |
|
|
|
90e13fc757
|
[Bugfix][HiSparse] Never allocate GPU pages without host backing (#59036)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-30 04:13:29 +00:00 |
|
|
Fanli LinandKunshang Ji
|
6a6c2952fa
|
[XPU][CI] Make test_mamba_prefix_cache block-size agnostic (#58269)
Signed-off-by: Lin, Fanli <fanli.lin@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-09-29 07:12:43 +08:00 |
|
|
|
924707f1bf
|
[CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs (#57237)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-27 05:01:31 -07:00 |
|
|
|
927c87b347
|
[PCP][DCP] Support DCP target model with non-DCP Dspark (#56723)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-26 18:12:35 -04:00 |
|
|
|
c0c35ddd8e
|
[Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-24 02:05:11 -07:00 |
|
|
Robert ShawandRobert Shaw
|
c64b15cde5
|
[Scheduler] Soften Long Prefill Tokens Threshhold (#57951)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
|
2026-09-21 17:19:42 -07:00 |
|
|
|
0b7f11a1ee
|
[AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-21 10:25:05 +00:00 |
|
|
|
038cb14bd7
|
[CI][Bugfix] Correct the Laguna DFlash acceptance-length reference (#57647)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-19 01:44:48 +00:00 |
|
|
Misha Goin
|
db7a24c230
|
[Perf] Add fused DFlash2 grouped convolution (#55960)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-17 21:38:59 +00:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
Li, Jiang
|
b68408043d
|
[Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-16 15:49:17 +08:00 |
|
|
Nick Hill
|
7dbc3d62c3
|
[Cleanup] Remove vestigial tpu_input_batch.py (#56898)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-14 21:29:15 -07:00 |
|
|
|
a5f6f61a8a
|
[CI][CPU] Add speculative-decoding coverage to CPU CI (#54934)
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-14 19:51:38 +08:00 |
|
|
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
|
|
6fe67cbbf3
|
[Spec][V2] Support MTP speculative decoding under pipeline parallelism (#46994)
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-11 17:55:21 +00:00 |
|
|
Wentao Ye
|
5fe77aecfc
|
[Deprecation] Deprecate items scheduled for 0.29 (#55353)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-11 08:17:02 -07:00 |
|
|
|
fadfe1c7d4
|
[Bugfix][Core] Retire Mamba states across null gaps (#55450)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-11 15:50:47 +08:00 |
|
|
Nick Hill
|
e6cb56337b
|
[Core] MRV2 support for fast-prefill (#56145)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-10 10:36:04 -07:00 |
|
|
|
d87a440f88
|
[Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514)
Signed-off-by: Yongqin Wang <yongqinwang@roblox.com>
Co-authored-by: Cursor <noreply@cursor.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-09-05 11:45:34 +08:00 |
|
|
Kevin H. Luu
|
76ba32160a
|
[CI] Shard CPU jobs above the 24h P90 threshold (#54751)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
|
2026-09-02 02:16:05 -07:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
87deddc7a4
|
[CI][Spec Decode] Add MTP placeholder-token regression coverage (#54893)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-02 03:29:38 -05:00 |
|
|
Andrii SkliarandAndrii Skliar
|
adebc41b7e
|
[Mamba] Add FlashInfer ReplaySSM backend (#52506)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-09-01 10:26:43 -04:00 |
|
|
Artem Perevedentsev
|
39e276eaeb
|
[Structured Output] Let terminal grammars stop under min_tokens (II) (#54218)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-08-31 20:39:27 +04:00 |
|
|
+5
|
e126687a9a
|
[Model] Support Qwen3.8-Flash-Next (#53896)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-31 05:57:56 +00:00 |
|
|
Giancarlo Delfin
|
d154d90d6c
|
[Model Runner V2] batch-sharded sample (#50465)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-24 15:49:57 -07:00 |
|
|
Misha Goin
|
a34f2ab70a
|
[Test] Add focused hybrid MTP prefix-cache regressions (#53189)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-21 17:05:39 -07:00 |
|
|
Morrison TurnanskyandTyler Michael Smith
|
1183f04b74
|
[Bugfix] test_batch_inference_correctness now uses batch invariance (#48040)
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-08-21 09:06:59 -07:00 |
|
|
|
76fb6d210a
|
[Bugfix][Core] Reserve the KV null block when validating max_model_len (#47272)
Signed-off-by: hyungjun oh <92hyungjun@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 02:49:23 +00:00 |
|
|
Nick HillandClaude Opus 4.8
|
615d4cfade
|
[Core] Check for GPU<->CPU syncs during CI (#43107)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 17:42:05 -07:00 |
|
|
Nick Hill
|
d6f17f3d53
|
[MRV2] Support attention-free models (#52374)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-14 23:54:50 +00:00 |
|
|
Nick Hill
|
513f83e7ec
|
[Bugfix] Take the sliding window from the layer, not the KV cache group (#51756)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-11 09:10:46 -07:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
1a1727330a
|
[CI] Solidify speculative decoding E2E coverage (#50713)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-10 19:56:39 -06:00 |
|
|
Nick Hill
|
9b0afeb4f6
|
[Perf] Avoid some more unnecessary GPU<->CPU syncs (#51458)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-08 17:13:26 -07:00 |
|