iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
18,665 Commits 3 Branches 0 Tags
8df14cfc8c8a09b4e57f082e59593a3abce4ffb3
Commit Graph
100 Commits
Author SHA1 Message Date
Lucas WilkinsonClaudeOpenAI CodexMatthew BonanniNicolò Lucchesi
 		51878e5b6e [2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends (#44455) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-07-11 11:11:16 -04:00
larryli2-amdandAndreas Karatzas c227aaa3f8 [ROCm] Enable DeepSeek-V4 DSpark speculative decoding on AMD (MI350X / MI355X, gfx950) (#47419) 
Signed-off-by: larryli2-amd <larryli2@amd.com>
Signed-off-by: larryli2-amd <Larry.Li@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-10 23:22:19 +08:00
Wentao Ye e257faf87d [Refactor] Remove unused rocm kernel combine_topk_swa_indices_ragged (#48158) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-10 09:33:07 -04:00
Wei ZhaoandYongye Zhu 2afa3f7e95 [Perf] Minimax M3 - Support cross-layer allreduce-norm fusion (#47631) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-07-07 21:16:32 -07:00
Jeff (Junze) MaandLucas Wilkinson 80eb01e93d [Bugfix] DSV4 TP16 garbage output (#47493) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-07-07 21:04:33 -07:00
Yongye ZhuandClaude Opus 4.8 9021589498 [Minimax-M3] Using tok_sparse_select from MSA instead of triton kernels (#47502) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-07 21:01:12 -07:00
Jason Li d99adcebdc [BugFix] Fix ModelOpt quantization inference for fused siblings (#47445) 
Signed-off-by: jasonlizhengjian <jasonlizhengjian@gmail.com>
 2026-07-08 03:19:42 +05:00
Jee Jee Li 5d23ca47ab [Kernel] Applies routed_scaling_factor internally (#47408) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-07-07 02:00:54 -07:00
Zhiyi Hongyy-fightingMichael Goin
 		04adc8843b [Bugfix]Fix DeepSeek-V4 fp8_ds_mla KV cache reshape (#47716) 
Co-authored-by: yy-fighting <23518844576@qq.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-07-06 12:56:44 -07:00
Wentao Ye f70caef48b [Perf] Cache token_to_req_indices for dsv4, 5x~6x kernel performance improvement (#47474) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-06 11:17:46 -04:00
Laurent-ZhangClaudemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		8d8ec38361 [Bugfix][Spec Decode] Add missing draft_id_to_target_id to DSparkDeepseekV4ForCausalLM (#47429) 
Signed-off-by: Laurent-Zhang <zhangdongsheng80@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-06 10:55:47 -04:00
Kunshang JiandHarry Mellor 07f9baf756 Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-06 14:18:33 +01:00
xiaozhoupy d039c17114 [Bugfix] Recycle post-final-norm hidden in GLM MTP (single norm) (#47448) 2026-07-06 01:07:56 -07:00
Giancarlo Delfin d63c8e9444 [BugFix][Spec Decode] Compact shared topk indices buffer after first MTP draft step (#47238) 2026-07-01 21:38:51 -07:00
Michael Goin 2b753ad200 [Spec Decode] DSpark speculators checkpoint support (#47093) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-01 17:32:27 -07:00
Benjamin ChislettGiancarlo Delfinmgoin
 		f5a8d73377 [Spec Decode] DSpark (#46995) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-07-01 08:30:24 -07:00
Fangzhou AiandClaude 4e5ca89cfe [ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing (#47269) 
Signed-off-by: Fangzhou Ai <fangzhouai@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-01 10:50:09 +00:00
akii96 aa8bb5562e [ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-07-01 17:33:55 +08:00
Fangzhou AiCursorClaude
 		ed41aa270a [ROCm][DSV4] Use aiter mHC pre/post as the default ROCm path (#43950) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Signed-off-by: Fangzhou-Ai <fangzhouai@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-01 16:27:42 +08:00
Jiangyun ZhuJingyi YangLucas WilkinsonClaude Opus 4.8
 		c5200d3565 [Attention][DSA] support dcp for FLASHINFER_MLA_SPARSE (#46076) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: GirasoleY <girasoleyang@gmail.com>
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-01 00:32:20 -04:00
Woosuk Kwon f098ee70c7 [GLM5] Support FlashMLA FP8 KV cache (Hopper & Blackwell) (#47090) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 18:13:21 -07:00
Kunshang JiandHarry Mellor e840f0d3f5 [Platform] Replace torch.cuda.Event with torch.Event (#47140) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 08:39:59 -07:00
Harry Mellor 1ab9522935 Remove more unnecessary load_weights methods (#47058) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 15:22:16 +01:00
Wentao Ye 75698e60b3 [Bug] Fix sparse attention issue for GLM5.2 non-torch compile path (#47083) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-29 15:45:53 -07:00
Ilya MarkovandMarkov Ilya a2abce646f [EPLB] Mask padding in EPLB load recording (#38128) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
 2026-06-28 19:43:58 -07:00
Harry Mellor 311ad689ad Remove boilerplate missed by #46820 (#46956) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-29 08:11:17 +08:00
Woosuk Kwon 4b643c463e [GLM5] Fix minor typo (#46961) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-28 08:37:00 -07:00
Woosuk Kwon 89876b0c54 [GLM5] Implement op fusion for GLM5/DSV3.2 (#46876) 2026-06-28 08:17:39 -07:00
Fangzhou AiClaude Opus 4.8tjtanaa
 		51a99565c3 [ROCm][Perf] Fused shared expert for Minimax M3 (#46474) 
Signed-off-by: Fangzhou-Ai <fangzhouai@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-06-27 12:34:17 +00:00
Woosuk Kwon 65e655d295 [GLM-5] Add DSV3.2/GLM5 to vllm/models/ (#46808) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-26 14:09:05 -07:00
Hongxia YangandClaude Opus 4.8 c2507fb293 [ROCm] [MoE] [Perf] Shared-expert fusion for bias-routed MoE; enable on MiniMax-M3 mxfp8 model (#46545) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-26 07:05:20 -07:00
qli88andTJian 8e394244a5 [ROCm]Enable AITER MoE backend for MiniMax-M3-MXFP4 (#46419) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-06-26 06:35:35 -07:00
Hongxia YangClaude Opus 4.8yueliu14tjtanaa
 		c63cd4906c [ROCm][ [Perf] sparse attention optimization on minimax-m3 (#46546) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: yueliu14 <yue.liu4@amd.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-06-25 16:56:00 +08:00
soaringk d7c1821b5a [Model][MiniMax-M3] Add pipeline parallelism support (#45810) 
Signed-off-by: soaringk <k3vin.zhang@gmail.com>
 2026-06-24 08:23:03 +00:00
Woosuk Kwonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0a3e2dbc09 [Optimization] Skip DP padding tokens in MoE (#46428) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-23 14:54:46 -07:00
Yongye ZhuClaude Opus 4.8Thien Tran
 		6691f087a6 [Minimax-M3] BF16/FP8 Indexer using MSA (#45892) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-06-23 10:28:49 -07:00
Gabriel WuClaude Opus 4.8Yongye Zhu
 		44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-22 11:54:14 -07:00
Martin Hickey ccd49f6821 [MyPy] Fix mypy for vllm/lora (#41722) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-22 10:57:09 -04:00
Tuukka Sarvi 89accad2cc [ROCm][DSV4] Disable TileLang MHC dispatch on gfx942 (#45931) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
 2026-06-22 09:26:54 +00:00
Nick Hill 7df3d7dada [Core] Ensure memory is pinned prior to async h2d copy (#45424) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-20 20:02:24 -07:00
Jeff (Junze) Maandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2a6c6b9429 [DeepSeek-V4] Support TEP=16 for the block-FP8 shared expert (#46001) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-18 20:10:12 -07:00
Jee Jee Li 22cc891108 [Kernel] Add PDL support for DeepGEMM kernel (#46006) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-18 20:49:01 +08:00
Tuukka SarviganyiCursorMarkus HartikainenJin Tao
 		afdcbd5d39 [ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681) 
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jin Tao <jintao12@amd.com>
 2026-06-18 12:21:14 +00:00
Micah Williamson e945169207 Revert "[Kernel] Add PDL support for DeepGEMM kernel" (#45999) 2026-06-17 22:59:48 -07:00
Jee Jee Li 4403af8fb5 [Kernel] Add PDL support for DeepGEMM kernel (#42996) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-17 20:37:17 -07:00
Woosuk KwonandClaude Opus 4.8 1797576237 Revert "[DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture" (#45309) (#45972) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-17 17:20:59 -07:00
qli88 d112eb1ac7 [feature] MiniMax-M3-MXFP4 support added (#45896) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2026-06-17 18:50:48 +00:00
Wentao Ye 2a47a9ff0f [DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture, 26.8% ~ 27.9% E2E TTFT improvement (#45309) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-17 09:34:53 -07:00
Wentao Ye 0a7bacdcac [DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-17 10:55:48 -04:00
Thien TranandYongye Zhu 4c62663315 [M3] Enable FP8 sparse GQA (#45744) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-16 21:38:03 -07:00