Wei Zhao
90d6d871d6
[Attention] Add CuTe DSL backend selection for MLA context prefill ( #62069 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-10-17 01:21:55 -04:00
24f632ca7e
[Bugfix] Engage indexer TP row sharding on total rows, not per-rank rows ( #61647 )
...
Signed-off-by: Zheng Cai <caizheng1993@gmail.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-15 13:54:14 +00:00
Maroon Ayoub and Claude
d6aca837c3
[Attention] Make the sparse MLA prefill workspace size configurable ( #61023 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 14:56:51 -04:00
Wei Zhao and Codex
03a7d3776f
[Perf] Support minimum KV splits configuration for TokenSpeed MLA decode ( #60345 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-09 23:33:47 +00:00
63b9931b73
[ROCm][DSv4.1] Paged MXFP4 sparse indexer on aiter's MQA-logits kernel ( #58671 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-29 14:37:57 +08:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Jared Wen and Claude Fable 5.1
1257512305
[DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer ( #56254 )
...
Signed-off-by: Jared Wen <w13431838023@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-15 14:07:14 -07:00
Nicolò Lucchesi
df42d112ee
[Config] Infer HiSparse attention config from HiSparseConnector ( #57041 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-09-15 19:44:26 +00:00
d43bb2f37f
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering ( #53781 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-12 00:36:47 -04:00
Wentao Ye
5fe77aecfc
[Deprecation] Deprecate items scheduled for 0.29 ( #55353 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-11 08:17:02 -07:00
Jee Jee Li and Cyrus Leung
4f686e182a
[MISC] Cleanup deprecated parameters ( #53559 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-24 10:19:04 -07:00
bbe8b23e1a
[Deprecation] Remove dead use_prefill_decode_attention flag ( #52557 )
...
Signed-off-by: Brian Woods <me@brian.biz >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-22 22:38:10 +00:00
Yongye Zhu and Claude Opus 5
967e104fad
[Config] Unify indexer cache dtype under attention_config.indexer_kv_dtype ( #52550 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-17 10:20:04 +08:00
Jason Li
0055b8bfa3
[Attention][MiniMax-M3] Add MSA speculative decode verification ( #50032 )
2026-08-02 13:46:55 -07:00
Andreas Karatzas
5b95890742
[FlexAttention] Avoid encoder block-mask compile explosion ( #50339 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 11:24:13 -04:00
Nicolò Lucchesi and Claude Opus 4.8
5784507da4
[Attention] Allow selecting a different attention backend per KV-cache group ( #48012 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-17 15:19:02 -04:00
Matthew Bonanni and Claude Opus 4.8
c4f5cd60da
[1/N] Add dense MHA path for sparse MLA short sequences ( #47327 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-14 00:29:56 +00:00
0a1c5034f5
[Model] Add MiniMax M3 support ( #45381 )
...
Signed-off-by: youkaichao <youkaichao@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-16 01:01:25 +08:00
Matthew Bonanni
9898f94abe
[Attention] Remove deprecated MLA prefill arguments ( #42555 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-05-14 10:34:06 -07:00
liangel-02
6b5c389ee3
expose flex block size for batch invariant mode ( #41252 )
...
Signed-off-by: Angel Li <liangel@meta.com >
2026-05-13 14:11:57 -07:00
f3fef12350
[Attention] Abstract the MLA prefill backends and eliminate cuDNN ( #32623 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-01 13:36:20 -04:00
Matthew Bonanni
fd74c90d9c
[Attention][Spec Decode] Allow independent drafter attention backend selection ( #39930 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-27 19:38:09 -07:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
f4b42df048
[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x capacity ( #38479 )
...
Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-04-14 19:57:13 -07:00
Matthew Bonanni
9c81f35b1a
[Attention][MLA] Re-enable FA4 as default MLA prefill backend ( #38819 )
2026-04-06 17:51:46 -04:00
Matthew Bonanni
2c734ed0e0
[Bugfix][MLA] Change default SM100 MLA prefill backend back to TRT-LLM ( #38562 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-03-30 09:51:24 -07:00
Wentao Ye
45bd5c8e75
[Mypy] Fix mypy for vllm/config ( #37808 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-03-23 14:33:59 +00:00
Matthew Bonanni
f444c05c32
[Attention] Use FA4 for MLA prefill ( #34732 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-03-12 12:10:17 -04:00
Nicolò Lucchesi
18e01a0a10
[Misc] Add --attention-backend auto option ( #35738 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-03-04 15:12:27 +00:00
8b5014d3dd
[Attention] FA4 integration ( #32974 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com >
2026-03-01 23:44:57 +00:00
Pavani Majety
578977bb5e
[SM100] Resubmit FMHA FP8 prefill for MLA ( #31195 )
...
Signed-off-by: Pavani Majety <pmajety@nvidia.com >
2026-02-10 16:18:43 -05:00
Harry Mellor
61e632aea1
Turn @config into a dataclass_transform ( #31541 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-02-03 17:40:59 +00:00
Wentao Ye
6437ff1fb9
[Deprecation] Remove deprecated environment variables ( #32812 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-01-22 02:25:16 +00:00
Matthew Bonanni and Wentao Ye
1a1fc3bbc0
[Attention][MLA] Make FLASHINFER_MLA the default MLA backend on Blackwell, and TRTLLM the default prefill ( #32615 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-01-19 18:41:34 -05:00
Matthew Bonanni
2e7c89e708
Revert "[Attention][MLA] Make FLASHINFER_MLA the default MLA backen… ( #32484 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-17 04:42:39 +00:00
Matthew Bonanni and Wentao Ye
8ebfacaa75
[Attention][MLA] Make FLASHINFER_MLA the default MLA backend on Blackwell, and TRTLLM the default prefill ( #32339 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-01-15 09:49:57 -05:00
Matthew Bonanni
2612ba9285
[1/N][Attention] Restructure attention: move files ( #31916 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-09 13:10:24 -08:00
66e674cdd5
[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments ( #26315 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2025-12-05 09:48:43 -08:00