iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
186 Commits
Author SHA1 Message Date
Yuwen ZhouandLi, Jiang <jiang1.li@intel.com> b346479065 [CPU] Add CPU FP8 W8A8 linear/MoE support (#49942) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-18 17:59:42 +08:00
Shanshan Shen bc0f47cd03 [ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 (#57132) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-16 18:58:23 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Nick Hill f2aad6aa70 [MRV2] Buffer util simplifications (#56888) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-09-15 20:36:51 +00:00
Ronald e0c04c7b4d [Bugfix][Models] Fix OpenPangu sleep mode with static sinks (#53696) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
 2026-09-14 21:05:47 +08:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		238cb2b191 [Perf][GLM-5.3-Flash] Dense/masked-MHA sparse prefill for the NoPE (256, 0, 256) layout + skip the NoPE K concat (#55738) 
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-14 09:36:39 +00:00
PatrykSafferpatryk.safferpatryk.safferLucas WilkinsonPatryk SafferClaude Fable 5.1
 		ebe1dec2da [PCP][DCP] Enable PCP+DCP on sparse-MLA models (#56157) 
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Signed-off-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Signed-off-by: PatrykSaffer <patryk.saffer@mistral.ai>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-1.blc1.mistralcompute.ai>
Co-authored-by: patryk.saffer <patryk.saffer@blc-login-0.blc1.mistralcompute.ai>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patryk Saffer <PatrykSaffer@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-12 23:24:44 +00:00
Matthew BonanniOpenAI CodexLucas WilkinsonLucas WilkinsonNicolò LucchesiOpenAI CodexClaude Opus 5Lucas Wilkinson
 		d43bb2f37f [3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-12 00:36:47 -04:00
Andy Lo dc07f1638f [Bugfix][MLA] Read sparse model settings from text config (#56160) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2026-09-11 18:02:13 +00:00
Chris EastwoodNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6fe67cbbf3 [Spec][V2] Support MTP speculative decoding under pipeline parallelism (#46994) 
Signed-off-by: Chris Eastwood <chris.eastwood@pwn4g3.dev>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-11 17:55:21 +00:00
Shanshan Shen 127143e27f Revert "[Rocm][Kimi-k3] Fix pipeline_parallel support for the kimik3 DCP mode (#53664)" (#56429) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-11 09:17:26 -07:00
Wentao Ye 5fe77aecfc [Deprecation] Deprecate items scheduled for 0.29 (#55353) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-11 08:17:02 -07:00
Matt b4da4d17ae [ROCm][Bugfix] Fix AITER preshuffled FP8 block-scale kernel (#56433) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-09-11 11:44:57 +00:00
Kevin H. LuuandOpenAI Codex 1cf6555214 [Bugfix] Pin EPLB and MLA host-to-device transfer buffers (#56138) 
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-11 01:57:47 -07:00
Jared WenandClaude Fable 5.1 1768273c13 [Perf][GLM-5.3-Flash] Decode hot-path cleanups: strided KDA recurrent inputs, NoPE MQA query without concat, no duplicate router GEMM (#55736) 
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-10 16:28:33 +08:00
Wei ZhaoandOpenAI Codex 8c87c333b8 [Perf] Fix TRTLLM ragged prefill perf regression (#55499) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-09 20:38:20 +00:00
haic0CursorShanshan Shen
 		d8d53f17c2 [Rocm][Kimi-k3] Add pipeline_parallel support for the kimik3 model (#53664) 
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-09 07:50:24 -07:00
bnellnmLucas WilkinsonOpenAI Codex
 		d29c88f162 [Core] Let SWA layers take the primary block size to avoid inflating the KV block LCM (#53007) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-08 16:44:43 -04:00
Jee Jee Li 199cb9b964 [Kernel] Remove unused fake implementation (#55535) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-09-06 17:54:32 -07:00
Nick Hill 8277c42e4c [Perf] Ensure async h2d copies are pinned in more places (#55202) 2026-09-04 17:17:01 -07:00
fxmarty-amdCodexmgoin
 		5690b02c03 [Quantization] Support online quantization with partially pre-quantized checkpoints (#51392) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-04 11:46:06 -07:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
Juhi MittalClaude Opus 4.8mgoin
 		7a977c0699 [ModelOpt] Redesign the LinearMethod classes using the generic QuantKey-driven method (#49381) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-01 16:28:56 -04:00
sychen52 7c5dc571cb [Attention][DSA] Enable W4A16 DSA (#51724) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-09-01 12:37:47 +08:00
Roberto L. CastroandCodex e16b5e518d [1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-31 23:26:05 -04:00
Nicolò Lucchesi 7f4793eaa3 [Nixl][PD] DCP support for MLA models (#50611) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-08-29 16:03:53 +00:00
Keyi LiKeyi LiClaude Opus 4.7
 		46a83642f6 [Perf] Pin CPU tensors before non_blocking H2D in three MM paths (#54292) 
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-28 20:13:40 -07:00
maobaolongandCodex 6f7df92a8e [CPU][MLA] Fix prefill backend selection so MLA runs end-to-end on CPU (#51471) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Codex <codex@openai.com>
 2026-08-28 12:36:55 +08:00
Matthew BonanniandOpenAI Codex de9250ac9e [Attention] Enable masked MHA for GLM-5 head dimensions (#53785) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-08-27 21:17:12 +00:00
cjackal 22099afc64 [Bugfix][DCP] Handle sparse MLA metadata after DCP Manager refactor (#52377) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-08-24 15:02:48 +00:00
Lucas Wilkinson 8bdc70ec7b [6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-08-21 17:47:49 -07:00
Wei ZhaoandOpenAI Codex f8e0602713 Support kimi k3 nvfp4 checkpoint (#53132) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 05:59:44 +00:00
Summer YangandWoosuk Kwon 58e5ee0158 [refactor] consolidate cp attn ops (#52839) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-19 18:04:11 -07:00
Roger Wang 755492e37d Revert "[Kernel] Gemma-4 FA4 FP8 Kernel" (#52987) 2026-08-19 10:48:46 -07:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 8e46accab2 [Attention] Vectorize sparse MLA mask loads (#52217) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 20:32:41 -07:00
Li, Jiang e4d61d0d22 [CPU] Add AMX-only high-performance MLA backend for DeepSeek V2/V3/R1 (#52616) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-08-19 10:28:52 +08:00
Woosuk KwonandOpenAI Codex 01e56caaf2 [Bugfix][MLA] Do not use Dense MHA for GLM-5.2 (#52512) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-18 11:20:48 -07:00
Wei Zhao d1e3eee6fb [Spec decode] Support Kimi-K3 DCP with DSpark (#52188) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-17 13:08:45 -07:00
Lucas WilkinsonandClaude 57bd0ed441 [5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-14 09:47:55 -04:00
Jhao-Ting ChenQixiang LinClaude Opus 4.8Yongye Zhu
 		827a2af806 [Kernel] Gemma-4 FA4 FP8 Kernel (#48666) 
Signed-off-by: Qixiang Lin <qixiangl@nvidia.com>
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Qixiang Lin <qixiangl@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-08-13 17:42:00 -07:00
Yongye ZhuClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		903d2efe7e [Attention][MLA] Fuse Kimi-K3 chunked-context K/V packing (#51772) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-12 22:42:57 -07:00
范裕达OpenAI CodexJee Jee LiJee Jee Li
 		9035151d6c [Model] Add native Dots3 NOTE multimodal support (#51255) 
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-08-12 23:13:20 +08:00
Wei Zhao 608c12473f [Attention] Fix MLA prefill workspace allocation size (#51733) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-08-11 05:34:30 +00:00
Yongye ZhuOpenAI CodexClaude Opus 5
 		c76a425278 [Kernel] Optimize long-context MLA cache gathers (#51739) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-10 22:23:43 -07:00
aoshen02andRoger Wang 99e62b802c [Bugfix][Model Loader] Defer post-load attention weight processing (#49519) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-10 19:16:57 -07:00
Summer YangClaude Fable 5云挚foraxeWoosuk Kwon
 		63ac04a61e [Kimi-K3] DCP support (#50484) 
Signed-off-by: girasoley <girasoley@inferact.ai>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: foraxe <73625538+foraxe@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-10 12:54:26 -07:00
Wei-Ming Chen 1c94e8dc7d Add NVFP4 KV 4-over-6 scale search (#45187) 
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
 2026-08-07 13:22:27 -07:00
Matthew BonanniClaude Opus 5OpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b38e111d3e [Attention][MLA] Per-request scheduling for MLA chunked context (#50613) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 16:45:44 +00:00
Jacob ZhangMatthew Bonannimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd930c8e78 [Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 (#38771) 
Signed-off-by: Jacob Zhang <jaco8123@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-05 07:25:41 -07:00