iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
164 Commits
Author SHA1 Message Date
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Juhi MittalClaude Opus 4.8mgoin
 		7a977c0699 [ModelOpt] Redesign the LinearMethod classes using the generic QuantKey-driven method (#49381) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-01 16:28:56 -04:00
sychen52 7c5dc571cb [Attention][DSA] Enable W4A16 DSA (#51724) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-09-01 12:37:47 +08:00
Roberto L. CastroandCodex e16b5e518d [1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-31 23:26:05 -04:00
Nicolò Lucchesi 7f4793eaa3 [Nixl][PD] DCP support for MLA models (#50611) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-29 16:03:53 +00:00
Keyi LiKeyi LiClaude Opus 4.7
 		46a83642f6 [Perf] Pin CPU tensors before non_blocking H2D in three MM paths (#54292) 
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-28 20:13:40 -07:00
maobaolongandCodex 6f7df92a8e [CPU][MLA] Fix prefill backend selection so MLA runs end-to-end on CPU (#51471) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-28 12:36:55 +08:00
Matthew BonanniandOpenAI Codex de9250ac9e [Attention] Enable masked MHA for GLM-5 head dimensions (#53785) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-27 21:17:12 +00:00
cjackal 22099afc64 [Bugfix][DCP] Handle sparse MLA metadata after DCP Manager refactor (#52377) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-08-24 15:02:48 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Wei ZhaoandOpenAI Codex f8e0602713 Support kimi k3 nvfp4 checkpoint (#53132) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 05:59:44 +00:00
Summer YangandWoosuk Kwon 58e5ee0158 [refactor] consolidate cp attn ops (#52839) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-19 18:04:11 -07:00
Roger Wang 755492e37d Revert "[Kernel] Gemma-4 FA4 FP8 Kernel" (#52987) 2026-08-19 10:48:46 -07:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8e46accab2 [Attention] Vectorize sparse MLA mask loads (#52217) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 20:32:41 -07:00
Li, Jiang e4d61d0d22 [CPU] Add AMX-only high-performance MLA backend for DeepSeek V2/V3/R1 (#52616) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-19 10:28:52 +08:00
Woosuk KwonandOpenAI Codex 01e56caaf2 [Bugfix][MLA] Do not use Dense MHA for GLM-5.2 (#52512) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-18 11:20:48 -07:00
Wei Zhao d1e3eee6fb [Spec decode] Support Kimi-K3 DCP with DSpark (#52188) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-17 13:08:45 -07:00
Lucas WilkinsonandClaude 57bd0ed441 [5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-14 09:47:55 -04:00
Jhao-Ting ChenQixiang LinClaude Opus 4.8Yongye Zhu
 		827a2af806 [Kernel] Gemma-4 FA4 FP8 Kernel (#48666) 
Signed-off-by: Qixiang Lin <qixiangl@nvidia.com>
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Qixiang Lin <qixiangl@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-13 17:42:00 -07:00
Yongye ZhuClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		903d2efe7e [Attention][MLA] Fuse Kimi-K3 chunked-context K/V packing (#51772) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-12 22:42:57 -07:00
范裕达OpenAI CodexJee Jee LiJee Jee Li
 		9035151d6c [Model] Add native Dots3 NOTE multimodal support (#51255) 
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-12 23:13:20 +08:00
Wei Zhao 608c12473f [Attention] Fix MLA prefill workspace allocation size (#51733) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-11 05:34:30 +00:00
Yongye ZhuOpenAI CodexClaude Opus 5
 		c76a425278 [Kernel] Optimize long-context MLA cache gathers (#51739) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-10 22:23:43 -07:00
aoshen02andRoger Wang 99e62b802c [Bugfix][Model Loader] Defer post-load attention weight processing (#49519) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-10 19:16:57 -07:00
Summer YangClaude Fable 5云挚foraxeWoosuk Kwon
 		63ac04a61e [Kimi-K3] DCP support (#50484) 
Signed-off-by: girasoley <girasoley@inferact.ai>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: foraxe <73625538+foraxe@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-10 12:54:26 -07:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
Matthew BonanniClaude Opus 5OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b38e111d3e [Attention][MLA] Per-request scheduling for MLA chunked context (#50613) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 16:45:44 +00:00
Jacob ZhangMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd930c8e78 [Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 (#38771) 
Signed-off-by: Jacob Zhang <jaco8123@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:25:41 -07:00
yimdevOpenAI CodexMatthew Bonanni
 		199644d410 [Bugfix][Attention] Guard sparse MLA masked MHA workspace (#50906) 
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-08-04 09:59:33 -04:00
wangxiyuan dd11df04f3 [Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-08-03 11:15:13 -04:00
Soila KavulyaandClaude aef85aed5d [Bugfix][TurboQuant] Add KV quant mode for turboquant (#50533) 
Signed-off-by: Soila Kavulya <soila.p.kavulya@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 17:42:32 +00:00
Matthew BonanniandOpenAI Codex 82ae4164ee [2/N][Attention] Enable masked MHA for sparse MLA prefills (#48770) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-31 06:01:24 -07:00
Martin Hickey b2fb83e7ff [Attention]: Use KVCacheSpec for AttentionMetadataBuilder type hints (#50148) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-31 11:20:18 +00:00
Richard ZouandCodex c27b080d26 [compile] Fix fake kernel return dtype (#50444) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-30 19:50:02 +00:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
yinfengLiu f1e8fd27ae [ROCm] Add AITER FP8 ViT encoder attention (#49937) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
 2026-07-30 06:22:10 +00:00
Yiliu DongandOpenAI Codex ba702e978e [Attention] Skip sparse indexer scoring for dense short prefills (#48407) 
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-28 16:17:22 +00:00
Colin ZTJianfanxingran
 		28158b2fc3 [ROCm] [BugFix] Fix Quark GLM-5.2 Checkpoint inference: indexer wk per-channel FP8 dequant + missing sparse-MLA metadata fields (#48886) 
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
Signed-off-by: ColinZ22 <Colin.Zeng@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: fanxingran <xingran.fan@amd.com>
 2026-07-27 22:38:22 +00:00
Lucas WilkinsonandOpenAI Codex 060b5f61dc [Bugfix][Attention] Ignore empty MLA context chunks during merge (#49294) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-22 02:23:48 +00:00
sungsoo haClaude Opus 4.8Lucas WilkinsonMatthew Bonanni
 		2396a61108 [Attention][MLA][DCP] Query replication for MLA decode (DeepSeek-V2/R1 + Kimi-K2.5) (#45964) 
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-20 23:51:27 +00:00
Lucas WilkinsonCodexOpenAI CodexMatthew Bonanni
 		b6ff8a2f50 [Core] Add MRV2 virtual-batch PCP for MLA (#46570) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-19 02:53:15 +00:00
aoshen02 17fdd42100 [Bugfix][Attention] Preserve post-load tensors across weight reloads (#48251) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-07-17 14:15:26 +08:00
Matthew BonanniOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		67f9046e4a [Bugfix] Sparse MLA: enable fp8_ds_mla dense prefill (#48642) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-16 22:44:04 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f17be06fbe [Perf] Optimize clamp to clamp_ (#48143) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-16 18:41:07 -04:00
Jack HuandDouglas Lehr 2cab53ddee [Model][Hardware][AMD]: Part 1/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_AITER_FA, and ROCM_AITER_UNIFIED_ATTN (#42749) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-07-16 17:39:04 -05:00
ruikangliuClaudeMatthew BonanniOpenAI Codex
 		0becb7486b [BugFix][MLA] Support kv_cache_dtype_skip_layers for MLA attention (#47309) 
Signed-off-by: liuruikang <liuruikang.cs@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-16 00:06:11 +00:00
drakoshaandMatthew Bonanni 6472131298 [Bugfix] Set kv_quant_mode on the generic MLA KV-cache spec (#48379) 
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-15 03:36:52 +00:00
Michael Goin b2f7d2560a [Bugfix] Make MLA+SWA check the layer's backend, not the model config (#48520) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-14 09:53:34 -07:00
Matthew BonanniandClaude Opus 4.8 c4f5cd60da [1/N] Add dense MHA path for sparse MLA short sequences (#47327) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-14 00:29:56 +00:00
Nicolò LucchesiClaude Opus 4.8Andreas Karatzas
 		bea70c7cfc [Attention] Make sliding-window support an explicit backend capability (#48011) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-13 01:07:56 -07:00