iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
18,363 Commits 3 Branches 0 Tags
2dfaae752b4db0d43cfc0715c780e33be030d0f1
Commit Graph
87 Commits
Author SHA1 Message Date
Giancarlo Delfin d63c8e9444 [BugFix][Spec Decode] Compact shared topk indices buffer after first MTP draft step (#47238) 2026-07-01 21:38:51 -07:00
Michael Goin 2b753ad200 [Spec Decode] DSpark speculators checkpoint support (#47093) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-07-01 17:32:27 -07:00
Benjamin ChislettGiancarlo Delfinmgoin
 		f5a8d73377 [Spec Decode] DSpark (#46995) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-07-01 08:30:24 -07:00
Fangzhou AiandClaude 4e5ca89cfe [ROCm][MiniMax-M3] Cross-layer lightning-indexer top-k sharing (#47269) 
Signed-off-by: Fangzhou Ai <fangzhouai@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-01 10:50:09 +00:00
akii96 aa8bb5562e [ROCm][Perf][Bugfix] DSv4 indexer: use platform FP8 dtype (fnuz) for Q-quant on gfx942 (#46730) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
 2026-07-01 17:33:55 +08:00
Fangzhou AiCursorClaude
 		ed41aa270a [ROCm][DSV4] Use aiter mHC pre/post as the default ROCm path (#43950) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Signed-off-by: Fangzhou-Ai <fangzhouai@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-01 16:27:42 +08:00
Jiangyun ZhuJingyi YangLucas WilkinsonClaude Opus 4.8
 		c5200d3565 [Attention][DSA] support dcp for FLASHINFER_MLA_SPARSE (#46076) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: GirasoleY <girasoleyang@gmail.com>
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-01 00:32:20 -04:00
Woosuk Kwon f098ee70c7 [GLM5] Support FlashMLA FP8 KV cache (Hopper & Blackwell) (#47090) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-30 18:13:21 -07:00
Kunshang JiandHarry Mellor e840f0d3f5 [Platform] Replace torch.cuda.Event with torch.Event (#47140) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 08:39:59 -07:00
Harry Mellor 1ab9522935 Remove more unnecessary load_weights methods (#47058) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-30 15:22:16 +01:00
Wentao Ye 75698e60b3 [Bug] Fix sparse attention issue for GLM5.2 non-torch compile path (#47083) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-29 15:45:53 -07:00
Ilya MarkovandMarkov Ilya a2abce646f [EPLB] Mask padding in EPLB load recording (#38128) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
 2026-06-28 19:43:58 -07:00
Harry Mellor 311ad689ad Remove boilerplate missed by #46820 (#46956) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-29 08:11:17 +08:00
Woosuk Kwon 4b643c463e [GLM5] Fix minor typo (#46961) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-28 08:37:00 -07:00
Woosuk Kwon 89876b0c54 [GLM5] Implement op fusion for GLM5/DSV3.2 (#46876) 2026-06-28 08:17:39 -07:00
Fangzhou AiClaude Opus 4.8tjtanaa
 		51a99565c3 [ROCm][Perf] Fused shared expert for Minimax M3 (#46474) 
Signed-off-by: Fangzhou-Ai <fangzhouai@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-06-27 12:34:17 +00:00
Woosuk Kwon 65e655d295 [GLM-5] Add DSV3.2/GLM5 to vllm/models/ (#46808) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-26 14:09:05 -07:00
Hongxia YangandClaude Opus 4.8 c2507fb293 [ROCm] [MoE] [Perf] Shared-expert fusion for bias-routed MoE; enable on MiniMax-M3 mxfp8 model (#46545) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-26 07:05:20 -07:00
qli88andTJian 8e394244a5 [ROCm]Enable AITER MoE backend for MiniMax-M3-MXFP4 (#46419) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-06-26 06:35:35 -07:00
Hongxia YangClaude Opus 4.8yueliu14tjtanaa
 		c63cd4906c [ROCm][ [Perf] sparse attention optimization on minimax-m3 (#46546) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: yueliu14 <yue.liu4@amd.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-06-25 16:56:00 +08:00
soaringk d7c1821b5a [Model][MiniMax-M3] Add pipeline parallelism support (#45810) 
Signed-off-by: soaringk <k3vin.zhang@gmail.com>
 2026-06-24 08:23:03 +00:00
Woosuk Kwonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0a3e2dbc09 [Optimization] Skip DP padding tokens in MoE (#46428) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-23 14:54:46 -07:00
Yongye ZhuClaude Opus 4.8Thien Tran
 		6691f087a6 [Minimax-M3] BF16/FP8 Indexer using MSA (#45892) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-06-23 10:28:49 -07:00
Gabriel WuClaude Opus 4.8Yongye Zhu
 		44d95069e9 Enable DeepSeek V4 and GLM-5.1 on SM120 (#43477) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-22 11:54:14 -07:00
Martin Hickey ccd49f6821 [MyPy] Fix mypy for vllm/lora (#41722) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-06-22 10:57:09 -04:00
Tuukka Sarvi 89accad2cc [ROCm][DSV4] Disable TileLang MHC dispatch on gfx942 (#45931) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
 2026-06-22 09:26:54 +00:00
Nick Hill 7df3d7dada [Core] Ensure memory is pinned prior to async h2d copy (#45424) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-20 20:02:24 -07:00
Jeff (Junze) Maandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2a6c6b9429 [DeepSeek-V4] Support TEP=16 for the block-FP8 shared expert (#46001) 
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-18 20:10:12 -07:00
Jee Jee Li 22cc891108 [Kernel] Add PDL support for DeepGEMM kernel (#46006) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-18 20:49:01 +08:00
Tuukka SarviganyiCursorMarkus HartikainenJin Tao
 		afdcbd5d39 [ROCm][DSv4] Functional fixes for DeepSeek V4 on MI300X/MI325X (#45681) 
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Jin Tao <jintao12@amd.com>
 2026-06-18 12:21:14 +00:00
Micah Williamson e945169207 Revert "[Kernel] Add PDL support for DeepGEMM kernel" (#45999) 2026-06-17 22:59:48 -07:00
Jee Jee Li 4403af8fb5 [Kernel] Add PDL support for DeepGEMM kernel (#42996) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-17 20:37:17 -07:00
Woosuk KwonandClaude Opus 4.8 1797576237 Revert "[DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture" (#45309) (#45972) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-17 17:20:59 -07:00
qli88 d112eb1ac7 [feature] MiniMax-M3-MXFP4 support added (#45896) 
Signed-off-by: Qiang Li <qiang.li2@amd.com>
 2026-06-17 18:50:48 +00:00
Wentao Ye 2a47a9ff0f [DSV4 Perf] Optimize dsv4 cudagraph by reducing eager_break_during_capture, 26.8% ~ 27.9% E2E TTFT improvement (#45309) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-17 09:34:53 -07:00
Wentao Ye 0a7bacdcac [DSv4 Perf] DSv4 flashinfer sparse index cache for metadata, 2%~4% TTFT improvement (#45863) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-17 10:55:48 -04:00
Thien TranandYongye Zhu 4c62663315 [M3] Enable FP8 sparse GQA (#45744) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-16 21:38:03 -07:00
Thien Tranandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 5bdc01bcc3 [M3] Tune Triton indexer score decode for spec-decode (#45743) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-16 21:07:34 -07:00
Cameron QuiliciHongxia YangTJian
 		efd15e192a [Bugfix][ROCm] Fix MiniMax-M3 FP8 KV cache dtype (#45720) 
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cameron Quilici <cjquilici@gmail.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-06-17 03:14:45 +00:00
Woosuk Kwon f4359a70f9 [DSV4][Minor] Fix supported KV cache dtypes (#44892) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-06-16 00:14:51 +00:00
Wentao Ye e18fe932ca [Perf] Optimize DSv4 prefill chunk planning, 4.0% E2E Throughput Improvement (#45061) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-15 19:50:21 +00:00
youkaichaoOpenAI CodexIsotr0pyThien TranBugen ZhaoJee Jee LiRoger WangfunctionstackxYongye ZhuJee Jee Li
 		0a1c5034f5 [Model] Add MiniMax M3 support (#45381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-16 01:01:25 +08:00
Ma Jianandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 04cec9e4d8 [XPU][DeepSeek-V4] Fix MTP: sync with upstream fixes #44821 and #43746 (#45240) 
Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 15:41:36 +08:00
Yufeng He 4673ca1d78 fix: prefix DeepSeek V4 MTP projections (#44821) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
 2026-06-10 08:47:04 -07:00
bnellnm f4966f8b3d [Bugfix] Fix weight loading issues caused by #41184 (#45054) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-06-10 01:20:13 -04:00
Wentao Ye d7607ad273 [Bug] Fix deepseek v4 OOM issue (#44914) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-09 15:47:06 -07:00
Ma Jian 70db1488c5 [DSV4][XPU] Add MHC fused_post_pre support (#44144) 
Signed-off-by: Ma Jian <jian1.ma@intel.com>
 2026-06-09 17:23:17 +08:00
Tyler Michael SmithandClaude Opus 4.6 e2f993dc41 [WideEP] Integrate DeepEP v2 (#41183) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-06-08 18:07:29 -07:00
Mohammad Miadh Angkad 823a0ab754 [Bugfix][MoE] Fix fused MoE expert mapping helper call sites (#44897) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-06-08 13:35:04 -07:00
bnellnm dc68bd8c41 [MoE Refactor] FusedMoE/MoERunner inversion refactor (#41184) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-06-08 10:42:58 -04:00