iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,825 Commits 3 Branches 0 Tags
f2376cdc4da852ec48ab5bcb747fdfa65a4dfd4a
Commit Graph
571 Commits
Author SHA1 Message Date
SeongJun Leeandlesj0610 a98247ab4d [Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 (#59853) 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Signed-off-by: lesj0610 <lesj0610@gmail.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-10-10 06:51:52 +00:00
Fangzhou AiandClaude Opus 5.5 193922d6ff [ROCm][Perf][DSv4.1] Mono decode layer for DeepSeek-V4.1-Flash on MI355X (#60397) 
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 20:53:18 -05:00
Dakai AnYifan QiaoOpenAI CodexOpenAI CodexLucas WilkinsonClaude Opus 5.5Lucas Wilkinson
 		02e55d2af3 [KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-09 15:48:37 -04:00
Warren Dengandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fc39317392 [Bugfix][Kernel] Remove invalid NC from Kimi-K3 KDA autotune keys (#60446) 
Signed-off-by: Warren Deng <warrdeng@meta.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-09 14:25:41 +00:00
Mikko TukiainenandCursor 81c993658e [ROCm][Perf] Fuse main QK-norm/RoPE/gate and KV-cache write into the AMD QSA prepare launch (#59732) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-09 21:14:49 +08:00
Chen ChengandIsotr0py d095e61c0b [Bugfix][Spec Decode] Fix GLM-5.3-Flash DFlash2/EAGLE3 aux hidden-state capture with the upstream config (#60763) 
Signed-off-by: Chen Cheng <85005591+ischencheng@users.noreply.github.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-10-09 11:35:48 +00:00
Warren Deng 59ffd73e85 [Bugfix][Kernel] Restore FP fusion for indexer Q quantization (#60716) 
Signed-off-by: Warren Deng <warrdeng@meta.com>
 2026-10-09 09:46:41 +00:00
Harry MellorandClaude Opus 5.5 2c99ee9333 [Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 09:29:03 +00:00
Chao Lei bc713cf8cc [Bugfix]MooncakeConnect add query timeout and retry (#55923) 2026-10-09 09:19:33 +00:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		efaca9547d [Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 (#59923) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-09 15:30:18 +08:00
Zheng CaizigzagcaiClaude Code
 		82e89e9944 [Perf][Qwen4Exp] Retune H200 M=4 merged QSA LL-GEMM plan (#60776) 
Signed-off-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: zigzagcai <zigzagcai@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
 2026-10-09 07:18:50 +00:00
Canlin GuoClaude Opus 5.5Juntian Liu
 		ad73a4740d [Perf][DSv4.1] Trim the last KV-source layer in decoder replay (#59894) 
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Juntian Liu <Juntianl777@gmail.com>
 2026-10-09 06:07:49 +00:00
Rohan PotdarandClaude Opus 5.5 88a419649e [ROCm][Kimi-K3] Use vLLM's FP8 max in the fused AttnRes quantization (#60717) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 04:45:13 +00:00
Yifan QiaoandClaude Opus 5.5 240785b82c [Bugfix] Keep the GLM-5.3 kpool tail and Qwen4 QSA ring out of the null block (#59528) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 20:44:04 +00:00
Shijin Zhang 2274e86842 [Spec Decode][Model] Support DFlash2 draft models with GLM-5.3-Flash (#56983) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
 2026-10-08 14:41:10 -04:00
kyleliang-nvKyle LiangOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1664c4f444 [Bugfix][MiniMax-M3] Guard sparse decode sentinel indices (#54597) 
Signed-off-by: Kyle Liang <kylliang@nvidia.com>
Signed-off-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: Kyle Liang <kyleliang-nv@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-08 18:27:57 +00:00
Elvir CrnčevićandMisha Goin 8674a794c7 [Perf] Enable Kimi K3 shared expert sharding with DeepEP v2 (#56875) 
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 16:42:15 +00:00
fxmarty-amdOpenAI CodexMisha Goin
 		9f3d70e081 [Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 12:04:46 -04:00
Mikko TukiainenandCursor e2414ec1db [ROCm][Qwen4Exp] Use fused PLE Triton kernels on the AMD backend (#60021) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-08 15:47:56 +00:00
kliuae d80ddaac73 [ROCm][Perf] Kimi-K3 Store only the current rank's shards in latent MoE up-proj (#59591) 
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
 2026-10-08 14:55:20 +00:00
Hexiang WangOpenAI CodexFangzhou Ai
 		458ba2edf8 [ROCm][Kimi-K3] Fuse AttnRes output with per-token FP8 input quantization (#59069) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-08 12:01:47 +00:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 02:30:52 -07:00
Nicolò Lucchesi 70bdce6133 [Model][GLM-5.3-Flash] Warm up the DCP top-k merge kernels for the kpool indexer (#60032) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-10-08 11:05:52 +02:00
JohnQinAMDClaude Opus 5Ren Yuzhoumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Douglas Lehr
 		fb2ac82458 [ROCm][DSv4.1][Perf] Replace the Torch prefill combine with the shared combine_topk_swa_indices kernel (#56638) 
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-08 14:15:18 +08:00
Hui ShiLuneExplorerArmageddonKnight
 		81198e97ba [Refactor] Abstract DeepGemm MegaMOE backend (#58053) 
Signed-off-by: Hui Shi <kalinshi@tencent.com>
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn>
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu>
 2026-10-08 05:05:39 +00:00
Hyukjoon LeeClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Summer Yang
 		1fbbfa1657 [Bugfix][Kimi-K3] Declare max_tp_shards on the DSpark MLA KV cache spec (#59733) 
Signed-off-by: hyukjlee <hyukjlee@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-10-08 00:33:32 -04:00
Isotr0pyandShanshan Shen 8352b2427f [Multimodal] Optimize encoder cudagraph padding logic (#58925) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-10-08 03:38:09 +00:00
v-shobhitClaude Opus 5.5Benjamin Chislett
 		8faac3dd2a [Bugfix] [Spec Decode] DeepSeek V4/V4.1 NVFP4: keep ignored MTP/DSpark experts on MXFP4 (#59655) 
Signed-off-by: Shobhit Verma <shobhitv@nvidia.com>
Signed-off-by: v-shobhit <161510941+v-shobhit@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
 2026-10-08 00:10:32 +00:00
Xuanteng HuangClaude Fable 5.1OpenAI Codex
 		f38c6778ce [quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740) 
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-07 22:01:09 +00:00
Robert ShawRobert ShawClaude Opus 5.5
 		e572e02d21 [UX] Skip checkpoint shards an MTP head does not load (#60152) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 15:16:19 -04:00
Daniel ChanjpvillamClaude Opus 5.5Rohan PotdarDaniel
 		d64a18832b [ROCm][Triton] Migrating MiniMax-M3 kernels from make_block_ptr to tensor descriptors (#59285) 
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Co-authored-by: jpvillam <juan.villamizar@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
 2026-10-07 11:49:59 -05:00
Micah WilliamsonandFangzhou Ai f6e3944e31 [ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 (#59685) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-07 09:35:53 -05:00
Jin TaoCursorSimon Danielsson
 		2a54f6b625 [ROCm][Perf][GLM-5.3-Flash] Enable AITER fused shared experts on gfx950 (#59221) 
Signed-off-by: Jin Tao <jin.tao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
 2026-10-07 20:53:40 +08:00
SeongJun Leelesj0610Thien Tran
 		d9503dc503 [Bugfix][Qwen4Exp] Derive the QSA head_dim from the unsharded head count (#59945) 
Signed-off-by: lesj0610 <lesj0610@godoiksan.org>
Co-authored-by: lesj0610 <lesj0610@godoiksan.org>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-07 12:24:40 +00:00
vovaniClaude Opus 5.5OpenAI CodexThien Tran
 		fe51194e2d [Bugfix][MiniMax-M3] Fence decode indexer shared-memory stage reuse (#60194) 
Signed-off-by: Vladimir Anisimov <vanisimov@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-07 01:35:12 +00:00
PatrykSafferandMisha Goin e43db1f5e2 Pad to 64 not 128 Q heads in flashMLA sparse (#60029) 
Signed-off-by: Patryk Saffer <patryk.saffer@mistral.ai>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 16:03:05 -04:00
aoshen02andClaude Opus 5.5 9a1f6fd4cb [Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading (#60064) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:20:09 +00:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		eb2901b735 [Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs (#59532) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 22:11:45 -07:00
Vadim GimpelsonClaudeThien Tran
 		b0b23a48d4 [Bugfix] Fix Qwen4Exp FP8 PLE pinned lookup kernel failing to compile below SM89 (#60142) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-06 01:56:44 +00:00
JohnQinAMDClaude Opus 5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		97bd8c74eb [ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache (#56720) 
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-05 17:57:46 -07:00
Brandon ZhangClaude Fable 5.1Wei Zhao
 		b2863c0de9 [Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path (#56881) 
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-10-06 00:13:55 +00:00
Vadim GimpelsonCursor Agentmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		0eac152707 [Bugfix] Route every speculation-capable row through the speculative path so recurrent state stays consistent (#56531) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 03:03:44 +04:00
Mikko TukiainenandCursor d23af12cb1 [ROCm][Perf] Reach the fused QSA pre-indexer from the AMD path (#57947) 
Signed-off-by: mjkvaak-amd <mjkvaak-amd@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-05 16:22:29 +00:00
Nicolò Lucchesi 30d4032363 [Model][GLM-5.3-Flash] Support DCP for the kpool sparse indexer (#59211) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
 2026-10-05 16:44:14 +02:00
cjackal 0e468adb43 [Model] Extend device-side mm normalization to Kimi K2.5 / K3 (#59278) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2026-10-05 21:28:13 +08:00
Stefano CastagnettaandThien Tran 4ff028d77e [Bugfix][Qwen4Exp] Honor --kv-cache-dtype-skip-layers in QSA attention (#60023) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-05 12:13:30 +00:00
Stefano CastagnettaandThien Tran 55b80221ed [Perf][Qwen4Exp] Keep the HC up projection on the skinny GEMM path (#60027) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-05 20:09:36 +08:00
Mikko TukiainenCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>TJian
 		edca360f13 [Bugfix][Qwen4Exp] Load PLE tables unquantized under Quark checkpoints (#59443) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-10-05 10:40:02 +00:00
sudhanshu shuklasudhanshu shuklaThien Tran
 		c04c79e50d [Performance] Add SM121 TP=2 skinny-GEMM plans (#59632) 
Signed-off-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com>
Co-authored-by: sudhanshu shukla <sudhanshu112233shukla@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-05 18:27:51 +08:00
bcsdhjewmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Codex
 		ae53b06898 [Bugfix][GLM-5.3-Flash] Keep batch x heads out of gridDim.z in the GLM-5.3-Flash fused recurrent KDA kernel (#56974) 
Signed-off-by: NolenLiang <nliang@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-05 09:31:45 +00:00