iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,772 Commits 3 Branches 0 Tags
89ac8d971be7d2f598598c1551a21c646bdf8b0d
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		2537629eb2 [Bugfix][KVConnector][NIXL] Size per-region replicate flags by each region's block count (#60107) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:27:50 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		0e8f8ecba8 [Bugfix][KVConnector][NIXL] Return NumPy descriptors from mixed-memory local registration (#60108) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-06 01:23:20 +00:00
+8
Lucas WilkinsonClaude Opus 5.5JartXTaneem IbrahimbnellnmItay EtelissimondanielssonRoberto L. CastroNetanel HaberItay AlroyFelix MartyVarun Sundar RabindranathMicah WilliamsoncjackalSummer YangRohan138kliuaeMisha Goin
 		7b6be2f1f4 [Docs] Add Reviewers page (#59459) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-02 16:11:31 +00:00
Lucas WilkinsonClaude Opus 5.5Matthew Bonanni
 		4056c8ac1f [Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool (#59309) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-01 23:04:21 -07:00
Lucas WilkinsonMatthew BonanniClaude Opus 5.5
 		90e13fc757 [Bugfix][HiSparse] Never allocate GPU pages without host backing (#59036) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 04:13:29 +00:00
Lucas WilkinsonClaude Opus 5Codex
 		927c87b347 [PCP][DCP] Support DCP target model with non-DCP Dspark (#56723) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-26 18:12:35 -04:00
Lucas WilkinsonQiuChunshuoClaude Fable 5.1CodexRobert ShawNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e6dc16cebd [PCP] Support prefill context parallelism with data parallelism (#57075) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 22:32:36 -04:00
Lucas WilkinsonandClaude Fable 5.1 5c121e962a [SpecDecode] Restore residual-logits comments in _resample_kernel (#58166) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-22 08:07:31 -07:00
Lucas WilkinsonandOpenAI Codex 8c0825090d [Bugfix][NIXL] Fix DCP pulls across MLA cache regions (#57389) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-21 11:57:49 -04:00
Lucas WilkinsonCodexRobert Shaw
 		62af3df734 [Bugfix][MLA] Reserve sparse prefill buffers before KV cache sizing (#57575) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-18 21:22:18 +00:00
Lucas WilkinsonandRobert Shaw fe284ea17d [BugFxi] Fix DeepGEMM FP8 workspace over allocation (#53914) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-09-17 09:35:34 -04:00
Lucas Wilkinson bd2d7ae7c1 [Perf] Reuse MoE workspace for DeepGEMM warmup (#57268) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-17 03:01:19 -04:00
Lucas WilkinsonandOpenAI Codex 38ca7a899c [Bugfix][HiSparse] Preserve per-layer offsets in KV cache bindings (#57027) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-15 20:27:11 +00:00
Lucas WilkinsonandClaude Opus 5 4be3dcf0fc [PCP][DCP] Declare FlashMLASparse MTP support at CP interleave > 1 (#56722) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-14 10:20:46 -04:00
Lucas WilkinsonandOpenAI Codex 586f652f8d [Bugfix][PCP][DCP] Respect interleave in indexer KV gather mapping (#56715) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-13 16:45:17 -04:00
Lucas WilkinsonandOpenAI Codex 7fe8fc803d [NIXL][PCP][DCP] Expose PCP producer KV shards as transfer ranks (#56645) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-13 20:23:15 +00:00
Lucas WilkinsonandOpenAI Codex b6e2aa748b [CI] Initialize warmup registry in V2 QSA runner fixture (#56707) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-13 13:34:09 -04:00
Lucas Wilkinson 1cfd972816 [CI] Fix NIXL transfer-rank geometry fixture (#56672) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-13 07:36:06 +00:00
Lucas Wilkinson 410f6da5c4 [Bugfix][KV Offload] Submit CPU stores on no-forward steps (#56621) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-09-13 05:56:14 +00:00
Lucas WilkinsonChao LeiClaude Fable 5Matthew BonanniOpenAI CodexOpenAI Codex
 		e19a3e172e [5/N] Share HiSparse host cache across TP ranks (reopens #52760) (#56629) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Chao Lei <leichao139636@163.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Chao Lei <leichao139636@163.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-12 19:48:25 -04:00
Lucas WilkinsonSummer YangOpenAI CodexOpenAI Codex
 		980c16c8e4 [PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. (#56107) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-10 22:07:53 -07:00
Lucas WilkinsonandNicolò Lucchesi 93911fcf68 [NIXL][PCP] Report replicated-PCP ranks > 0 as done sending instead of hiding them (#53903) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
 2026-09-10 12:58:05 +02:00
Lucas Wilkinson 5707355209 [Bugfix][MLA] Fix BLHNC addressing for FlashInfer sparse MLA (#54465) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-31 01:13:42 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Lucas WilkinsonandOpenAI Codex 0a3a4f7f35 [Bugfix][KV Connector][NIXL] Support PCP producers (#52779) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 16:22:01 -04:00
Lucas WilkinsonOpenAI CodexClaude Opus 5
 		72aedcc426 [DCP] Default query replication for GLM sparse attention (#50382) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 09:07:05 -07:00
Lucas WilkinsonandOpenAI Codex 5df31ea52d [Spec Decode] Enable adaptive verification on DSv4 + sm90 (#52795) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-20 22:44:55 -04:00
Lucas WilkinsonClaude Opus 5Robert Shaw
 		75dde08d3f [Perf][MoE] Optimize deepep_v2 receiver CPU Overhead (#51114) 
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-08-17 18:21:32 +00:00
Lucas WilkinsonOpenAI CodexYongye Zhu
 		292187dd8c [Bugfix][DSv4] Keep indexer scoring in breakable graphs (#52492) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-17 03:01:13 +00:00
Lucas WilkinsonandClaude 57bd0ed441 [5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-14 09:47:55 -04:00
Lucas Wilkinson d4c24e6f5d [CI] Increase extended generation test timeout (#52252) 2026-08-14 04:23:51 -04:00
Lucas WilkinsonandOpenAI Codex 79f3183f86 [Bugfix] Bound KV block zeroing launch geometry (#52058) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 03:42:26 +00:00
Lucas WilkinsonOpenAI CodexClaude Opus 5Benjamin Chislettmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Nick HillOpenAI Codex
 		7f7a32cfec [Spec Decode] DSpark confidence-scheduled verification (#47808) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-12 12:27:48 -07:00
Lucas WilkinsonandOpenAI Codex 02ac17851d [Refactor][MRV2] Unify uniform decode token count helper (#51917) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-12 05:58:43 +00:00
Lucas WilkinsonandClaude 61874f9842 [4/N][KV-Cache Layout Refactor] Promote local KV cache specs via a class-changing replace helper (#51612) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-11 17:29:29 -04:00
Lucas Wilkinson 789c4f905e [CI] Bump CUTLASS DSL to 4.6.2 (#51566) 2026-08-10 09:06:08 -04:00
Lucas Wilkinson 021b7d985b [Perf] Raise Blackwell CUDA graph capture default to 1024 (#49390) 2026-08-07 15:18:14 -04:00
Lucas Wilkinson 60417b4b74 [Core][PCP] Select MRV2 when PCP is enabled (#50034) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-07-27 17:03:26 -07:00
Lucas WilkinsonandOpenAI Codex 060b5f61dc [Bugfix][Attention] Ignore empty MLA context chunks during merge (#49294) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-22 02:23:48 +00:00
Lucas WilkinsonClaudeOpenAI Codex
 		6700813f86 [3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions (#44456) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-21 09:16:15 +00:00
Lucas WilkinsonCodexOpenAI CodexMatthew Bonanni
 		b6ff8a2f50 [Core] Add MRV2 virtual-batch PCP for MLA (#46570) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-19 02:53:15 +00:00
Lucas WilkinsonClaudeOpenAI CodexMatthew BonanniNicolò Lucchesi
 		51878e5b6e [2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends (#44455) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-07-11 11:11:16 -04:00
Lucas Wilkinson bec0a4ede6 [Revert] [Build] Update vllm ...builds FA3 with torch stable API (#48269) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-07-11 05:20:25 +00:00
Lucas Wilkinsonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 04d553f390 [Misc] Use meta tensor for KV cache stride calculation (#47316) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-10 23:24:59 -04:00
Lucas Wilkinsonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9c18e90f6c [BugFix] Fix packed HND KV cache reshape for FlashAttention (#47314) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-10 23:22:39 -04:00
Lucas Wilkinson e12b91b032 [CI] Fix cargo-deny config flag ordering (#48170) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-07-09 21:43:58 +00:00
Lucas Wilkinson ea0fa34f49 [CI] Increase extract hidden states TP2 timeout (#48161) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-07-09 16:19:03 -04:00
Lucas WilkinsonandCodex e08a915146 [Bugfix] Preserve tensor causal metadata for grouped attention (#48135) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-09 17:57:53 +00:00
Lucas Wilkinson fa4321de3d [Bugfix][TurboQuant] Preserve KV cache dtype in backend shape (#47609) 2026-07-05 08:20:48 +00:00
Lucas WilkinsonOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		27d5f78b63 [CI] Move distributed small LM eval to B200 (#47048) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-30 13:34:25 -04:00
Lucas Wilkinson e7df232288 [KV Offload] Gate packed HMA KV cache on cross-layer config (#46252) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-06-24 11:55:30 -04:00
Lucas WilkinsonOpenAI CodexTyler Michael Smith
 		cc22621b51 [KV Offload] Support packed HMA KV cache layout (#46205) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-06-20 21:19:40 +00:00
Lucas WilkinsonMartin KuklaMatthew BonanniDipika SikkaNickLucchejiahancAlec KohlhoffPorras HuangClaudescoootscooob
 		eb28452b10 [Model] Add DiffusionGemma Support (#45163) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
 2026-06-11 22:17:35 -07:00
Lucas WilkinsonClaude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Matthew Bonanni
 		4dcd10eb0d [1/N][KV-Cache Layout Refactor] Refactor DSV4 KV cache config construction (#44454) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-06-07 14:53:37 +00:00
Lucas Wilkinson 4721bb3aa4 [MRV2] Remove Eagle's dedicated CUDA graph pool (#44078) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-05-31 22:00:33 -07:00
Lucas WilkinsonMatthew BonanniNicolò Lucchesi
 		7e33081cee [Attention] Make FlexAttention and FlashAttention use num-blocks first layouts (#42095) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-26 19:55:56 -07:00
Lucas Wilkinson b1728c1e66 [Attention][Cleanup] Remove tree attention (#42121) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-05-08 18:36:19 -07:00
Lucas WilkinsonandRobert Shaw f83de7196f [BugFix] Fix OOB read in CUTLASS grouped GEMM with epilogue (#38571) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-09 23:52:52 -04:00
Lucas Wilkinson 70406eb1dc [Attention][V0 Deprecation] Deprecate accept output buffer (#39125) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-07 17:14:58 -04:00
Lucas Wilkinson 47e605092b [Gemma4] Enable Fast Prefill Optimization (#38879) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-06 11:19:39 -04:00
Lucas Wilkinson cb3935a8fc [FA4] Update flash-attention to latest upstream FA4 (#38690) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-02 17:02:37 +00:00
Lucas Wilkinson eb47454987 [Bugfix][MLA] Add logits size budget to sparse indexer prefill chunking (#36178) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-04-01 00:15:53 -04:00
Lucas Wilkinson e1d85e5c24 [Attention] Support distinguishing between short extends and decodes (#37303) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-03-20 10:49:36 -07:00
Lucas Wilkinson 483463f735 [MRV2] Extensible CG dispatch rework (#35959) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-03-09 13:58:45 -07:00
Lucas WilkinsonandMatthew Bonanni 28ef9ba399 [BugFix] Add support for MTP num_speculative_tokens > 1 with sparse MLA (#34552) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-03 07:21:57 -08:00
Lucas WilkinsonandWentao Ye f44d1ddc8c [BugFix] Fix cmake based incremental install (wrong vllm install dir) (#35773) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-02 21:58:16 -08:00
Lucas WilkinsonMatthew BonanniTyler Michael Smith
 		8b5014d3dd [Attention] FA4 integration (#32974) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2026-03-01 23:44:57 +00:00
Lucas Wilkinson 1d532f9d8f [DP] Only use DP padding when cudagraphs are actually used (#34102) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-27 15:14:31 -05:00
Lucas Wilkinson 5e58bdc711 [Bugfix] Remove erroneous lower bound on LoRA vocab size constraint (#35354) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-26 18:44:50 +00:00
Lucas Wilkinson 9511a3f8ee [Bugfix] Fix AttributeError in SMControlContextManager (#35338) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-25 18:01:10 -08:00
Lucas Wilkinson 15d76f74e2 Revert "[Misc] Enable weights loading tracking for quantized models" (#35309) 2026-02-25 09:20:15 -08:00
Lucas Wilkinson 542ca66357 Revert "[CI/Build] Remove redundant OpenTelemetry pip install from CI configs" (#35211) 2026-02-24 09:26:42 -08:00
Lucas Wilkinson 0e22cd618b Revert "[Llama4,Quantization] Simplify and generalize logic for Q/K permutations in quantized self-attn layers " (#34997) 2026-02-20 17:19:19 -08:00
Lucas Wilkinson aaefc58ee0 [CI] Revert PRs 34818 and 33600 (#34979) 2026-02-20 13:25:50 -08:00
Lucas WilkinsonandJee Jee Li bb85929aa6 [BugFix] Fix Python 3.13 FlashMLA import error (#34548) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-02-15 20:09:18 -08:00
Lucas WilkinsonandBill Nell 136b0bfa59 [BugFix] Fix DP chunking (#34379) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Bill Nell <bnell@redhat.com>
 2026-02-12 06:44:03 +00:00
Lucas Wilkinson c7914d30f9 Reapply [Attention][FA3] Update FA3 to include new swizzle optimization (#34043) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-11 07:07:56 -08:00
Lucas Wilkinson ba0511fd80 [Misc] Add run one batch script that supports profiling (#32968) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-10 18:29:49 -08:00
Lucas WilkinsonandBenjamin Chislett 81e217fe6b [Bugfix] Fix DP Attention Padding in Dummy Run (#34187) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-02-10 05:29:39 +00:00
Lucas WilkinsonandMatthew Bonanni d0d97e2974 [Misc] Fix up attention benchmarks (#33810) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-02-09 09:42:03 -05:00
Lucas Wilkinson 0e92298622 [Misc] Delay deprecation of CommonAttentionMetadata properties (#33801) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-04 08:41:57 -08:00
Lucas Wilkinson 2267cb1cfd [Attention][FA3] Update FA3 to include new swizzle optimization (#23465) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-02-03 08:08:47 -08:00
Lucas Wilkinsongemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Michael Goin
 		0a3c71e7e5 [BugFix] Fix whisper FA2 + full cudagraphs (#33360) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-01-31 12:15:06 +08:00
Lucas Wilkinson a650ad1588 [Misc] Remove missed pad_for_cudagraph (#33283) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-29 09:12:05 +00:00
Lucas Wilkinson 566cdb6cfb [CI] Fix MHA attention test failure (AttributeError when model_config is None in ViT attention backend) (#33033) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-25 19:49:53 -08:00
Lucas Wilkinson da5e7b12be [MLA] Fuse cat and qaunt for fp8 kv-cache (#32950) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-24 16:03:02 +00:00
Lucas Wilkinson 3a41459501 [cudagraphs] Refactor cudagraph capture loop (#32946) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-23 13:22:20 -07:00
Lucas Wilkinson 889722f3bf [FlashMLA] Update FlashMLA to expose new arguments (#32810) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-21 22:02:39 -07:00
Lucas Wilkinson b4f64e5b02 Update FlashMLA (#32491) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-21 13:03:37 +08:00
Lucas WilkinsonandMatthew Bonanni 2261340806 [Misc] Remove pad_for_cudagraphs from config (#30143) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-20 15:05:48 -05:00
Lucas Wilkinson ca21288080 [CI] Fix OOM in Hopper Fusion E2E Tests (H100) (#32489) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-16 21:27:16 +00:00
Lucas Wilkinson 14ce524249 [CI] Breakup h200 tests (#30499) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-16 06:23:22 +00:00
Lucas Wilkinsonandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> c36ba69bda [BugFix] Fix assert x_s.shape[-1] == x_q.shape[-1] // group_shape[1] in Blackwell Quantized MoE Test (#32362) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-15 10:19:12 -08:00
Lucas WilkinsonandEldar Kurtić 2c9b4cf5bf [BugFix] Fix DeepSeek-V3.1 + DeepGEMM incompatible scale shapes (#32361) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Eldar Kurtić <8884008+eldarkurtic@users.noreply.github.com>
 2026-01-15 06:32:22 +00:00
Lucas Wilkinson da6709c9fe [Misc] Delay deprecation of CommonAttentionMetadata properties (#32074) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-09 21:06:44 -08:00
Lucas Wilkinson 0a0aa07747 [Quant] Make static quant support all group shapes (#30833) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-09 12:49:27 -08:00
Lucas WilkinsonandLuka Govedič 6cdf015c3c [Misc] Fix Current vLLM config is not set. warnings, assert to avoid issues in the future (#31747) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 15:20:49 -08:00
Lucas Wilkinson be6a81f31b [chore] Update FA commit (#30460) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-07 23:24:18 -08:00
Lucas Wilkinson c7a79d41a0 [Attention][3/n] Remove usage of deprecated seq_lens_cpu and num_computed_tokens_cpu CommonAttentionMetadata properties (#31850) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-01-07 13:31:34 +08:00
Lucas WilkinsonandCyrus Leung 4c73be14e0 [Attention][2/n] Remove usage of deprecated seq_lens_cpu and num_computed_tokens_cpu CommonAttentionMetadata properties (#31774) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-06 17:32:14 +00:00