iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,112 Commits 3 Branches 0 Tags
4e59d53ab534fbb4aaa751eb37ecfbcf438855b5
Commit Graph
474 Commits
Author SHA1 Message Date
Muhammad Ahmed 0af34418e9 [ROCm][DSv4.1][Perf] Run the delayed mHC seams through aiter's fused Triton kernel (#58655) 
Signed-off-by: Ahmed <Muhammad.Ahmed@amd.com>
 2026-09-29 10:19:10 +08:00
Mehmet Cagri 70ae0b7435 [ROCm][MLA] Enable sparse MLA Gluon kernel from Aiter (#53492) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Signed-off-by: Mehmet Cagri <mehmet.kaymak@amd.com>
 2026-09-29 09:08:18 +08:00
rmhaskarJarrel SeahTyler Michael SmithOpenAI Codex
 		e03875f4d2 [Minimax M3] Enable fp8 indexer cache on triton indexer for non-SM100 architectures. (#59081) 
Signed-off-by: Jarrel Seah <jarrel@harrison.ai>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Rajas Mhaskar <rmhaskar@nvidia.com>
Co-authored-by: Jarrel Seah <jarrel@harrison.ai>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 19:21:13 -04:00
Oxana KorzhandClaude Opus 5.5 94d1462924 [Bugfix][Kimi-K3] Refresh DSpark context KV cache pointers after the KV cache is re-bound (#58814) 
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-28 11:05:10 -05:00
xaguilar-amdandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a7ff44355c [ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP (#54956) 
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-28 10:31:29 -04:00
Shanshan Shen 7ba3df63cb [ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform (#58983) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-28 05:46:28 -07:00
Jiangyun ZhuandOpenAI Codex 20b52e9f5b [Perf][Qwen3.8] Reduce PLE metadata construction overhead (#58114) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 20:46:22 +08:00
BaoYunkai f6877f563c [Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek V4 MTP drafter (#57568) 
Signed-off-by: BaoYunkai <ybao@amd.com>
 2026-09-28 20:16:38 +08:00
Thien Tran d6cce94fd4 [Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA (#58957) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-28 18:51:38 +08:00
Gabriel WuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		a2be4d3cc7 [Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views (#58961) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-28 08:11:46 +00:00
Thien TranandCodex 55de40a2fc [Minimax-M3] Add Encoder CUDA graph support (#58673) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <codex@openai.com>
 2026-09-28 04:17:36 +00:00
Shanshan Shen 44af287ebe [ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash (#57407) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-27 23:48:01 +08:00
Juntian LiuandClaude Opus 5.5 a9eafde59c [Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 (#58634) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-26 21:16:12 -07:00
Thien Tran 7a877ae34d [Qwen3.8-Flash-Next] Avoid memory fragmentation in QSA indexer logits workspace (#57105) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-27 10:11:52 +08:00
Yongye ZhuandClaude Opus 5.5 77871126f9 [Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 03:08:36 +08:00
Kevin H. Luu 7d8c5fe9a9 [Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#58499) 2026-09-26 11:37:36 -04:00
Wentao Ye 3576691c42 [GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 11:35:28 -04:00
qizixiOpenAI CodexClaude Opus 5.5Nick Hill
 		a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-26 17:01:17 +08:00
xiao-llmandCursor 5840d95284 [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071) 
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-25 20:51:18 -05:00
Shuolei Wang 4bb804cc5b [Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678) 
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
 2026-09-26 01:15:57 +00:00
jiacao-amdOpenAI CodexAndreas Karatzas
 		1417022c3d [ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045) 
Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 12:33:30 -05:00
Matt MastracciYifan QiaoClaude Opus 5.5Robert Shawmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2617fe9383 [Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#58454) 
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 12:51:34 -04:00
Yongye ZhuandClaude Opus 5.5 378504a544 [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 15:38:29 +08:00
Yongye ZhuandClaude Opus 5.5 5ff3bbfb08 [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 23:26:50 -07:00
Mathew Oddenopencode+deepseek-v4-flash+vllmAndreas KaratzasOpenAI Codex
 		267eee54bf [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) 
Signed-off-by: Mathew Odden <modden@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-24 23:28:20 -05:00
Juntian LiuKimi CodeClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		6936e77ba4 [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:52:02 +08:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		48d8880d09 [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 10:51:50 +08:00
Juntian Liu 609731e8f5 [Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path (#57679) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
 2026-09-25 01:39:03 +00:00
Thien TranKimiIsotr0py
 		16070ed7e9 [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-24 18:30:32 -07:00
Yifan QiaoandClaude Fable 5.1 39724758e7 [Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 00:13:55 +00:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Fangzhou AiandClaude Opus 5.5 721d0e5c11 [ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM (#58456) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 11:58:14 +00:00
Thien TranandOpenAI Codex 00b7847c80 [Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512) 
Signed-off-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-24 00:41:36 -07:00
Wei ZhaoCodexYongye ZhuKevin H. Luu
 		44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-09-23 22:45:23 -07:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
 2026-09-23 20:29:06 -07:00
Andreas Karatzas 56b3acb55b [ROCm][Bugfix] Keep zero MiniMax MXFP8 activation blocks finite (#58089) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 16:42:33 +08:00
aoshen02andYongye Zhu 973a3be780 [Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions (#57732) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-23 01:16:20 -07:00
Albert Chengmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Benjamin Chislett
 		88aa0d287d [Spec decode] Support variable-length decode for Kimi-K3 adaptive ver (#52988) 
Signed-off-by: Albert Cheng <albecheng@nvidia.com>
Signed-off-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-bia01.bia.clusters.nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-22 20:43:32 -07:00
Fangzhou Ai f9dce295c9 [ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57451) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
 2026-09-23 01:23:55 +00:00
Jack HuJames E T SmithDouglas LehrDouglas Lehr
 		1c0eee919d [ROCm][Model][Bugfix] Enable GLM-5.2-MXFP4 on the deepseek_v32 path and fix sparse attention correctness (#51915) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Signed-off-by: Jack Hu <jack.hu@amd.com>
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: James E T Smith <jamesETsmith@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
 2026-09-22 18:08:51 -05:00
lucamotz 276db94fb2 [Bugfix][Model][Spec Decode] Defer disposable GLM MTP head (#55442) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
 2026-09-22 20:08:27 +00:00
Fangzhou AiCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ca831d1c55 [ROCm][DSv4.1][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57435) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-22 18:05:11 +00:00
Juntian LiuClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		a6131b0e65 [Bugfix][Engram] Fall back when /dev/shm is absent before sharing tables (#57914) 
Signed-off-by: Juntian Liu <juntianl@inferact.ai>
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 00:19:09 +08:00
Jared WenClaude Fable 5.1mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c9b34fdb2d [Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard (#58061) 
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-22 21:58:21 +08:00
Canlin GuoThien TranOpenAI CodexClaude Fable 5.1Jee Jee Li
 		f92b78f6ef [Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter (#57428) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-22 01:07:17 -07:00
Shanshan Shen 42a85a4976 [ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 (#57919) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
 2026-09-22 04:45:53 +00:00
yinfengLiuLiuyinfeng01Andreas Karatzas
 		382970ee6c [Kimi-K3][AMD] Return KDA and MLA projection outputs directly (#50592) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-21 15:44:41 -05:00
Tuukka Sarviandlarryli2-amd e34685dfc0 [ROCm][DSv4] Enable DSpark adaptive verification (#52362) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Signed-off-by: larryli2-amd <larryli2@amd.com>
Co-authored-by: larryli2-amd <larryli2@amd.com>
 2026-09-21 10:03:41 -05:00
Fangzhou AiandCursor 04c1f4a407 [Bugfix][ROCm][DSv4.1] Disable SWA bounded replay on ROCm (#57906) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-21 10:29:14 +00:00
weitliaoCursorShanshan Shen
 		b8cf275382 [AMD][Minimax-M3][perf] Enable packed LBHNC AITER QK-norm fusion for MiniMax-M3 on ROCm (#54535) 
Signed-off-by: Wei-Ting Liao <wei-ting.liao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-21 16:38:30 +08:00