iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
1657 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
MOVZX c5ab42bc0f Merge branch 'main' into patches 2026-10-18 16:04:36 +07:00
Andreas KaratzasAndreas KaratzasClaude
 		da9134bc89 [ROCm][CI] Give the MoE parallel-launch watchdog room for AITER JIT builds (#62161) 
Signed-off-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <AndreasKaratzas@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-17 21:49:27 -05:00
afriedriSimon DanielssonKevin H. LuuAndreas Karatzas
 		2067297b2f [ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var (#61278) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 04:00:28 -05:00
Jee Jee Li e56b1603af [Kernel]Add norm fused kernel for Step-5 Preview and Muse Spark Open (#62132) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-17 01:53:00 -07:00
MOVZX e0fe5df731 Merge branch 'main' into patches 2026-10-17 08:06:56 +00:00
Jee Jee Li 39983a5e7a [Kernel] Add FP32 router GEMM support for Step-5 Preview and Muse Spark Open (#62126) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-16 22:17:55 -07:00
Vadim GimpelsonandClaude 783380af39 [Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 (#61718) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-16 17:24:51 -07:00
Woosuk KwonLainCodexClaude Opus 5.5vllm-agent
 		596e530675 [Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm (#62042) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Lain <fusiyuan2000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 15:39:46 -07:00
Tyler Michael SmithCodexRobert Shaw
 		68ebea789a [Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts (#59044) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 16:20:52 -04:00
Guan-Ming Chiuandvllm-agent 3dc6def360 [Attention][Spec Decode] Enable varlen Mamba2 decode for adaptive verification (#53036) 
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
 2026-10-16 11:23:59 -07:00
Misha GoinClaude Opus 5.5Kevin H. Luu
 		cffc0cfa31 [CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup (#61695) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-16 12:00:12 -04:00
Wentao Ye 8e25bbca19 [Refactor] Remove dead code multiple places (#61843) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-16 15:01:07 +00:00
MOVZX be95edb7f6 Merge branch 'main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-16 17:35:42 +07:00
Yicong WangOpenAI CodexLucas WilkinsonClaude Opus 5.5Kevin H. Luu
 		fa75d3456d [Feature][PCP] Add DeepSeek-V4.1-Flash support (#59857) 
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 23:17:51 -07:00
Andreas KaratzasandClaude b466fbacd2 [ROCm][Perf] Any-shape low-latency MoE router GEMM on MI355 (#61895) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-15 23:22:31 -05:00
Jack HuandDouglas Lehr ef38e8d006 [Model][Hardware][AMD][Kernel] Part 2/2 -> Enable e2e QK Norm + RoPE + KV Cache runtime fusion for Qwen3-30B-A3B on ROCM_ATTN (#42755) 
Signed-off-by: Jack Hu <Jack.Hu@amd.com>
Signed-off-by: Jack Hu <jingxujaster@hotmail.com>
Signed-off-by: Jack Hu <jack.hu@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-15 17:10:14 -07:00
Juan VillamizarJaden MathiasjpvillamOmar MuhammadDanielClaude Opus 5Daniel Chan
 		e711723352 [ROCm] gfx1250 optimizations and model enablement (#58800) 
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
 2026-10-15 14:51:57 -07:00
Mikko Tukiainen 0c25849292 [ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K (#59406) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
 2026-10-15 21:11:33 +00:00
Juntian LiuClaude Opus 5.5Kimi CodeYifan Qiao
 		f8460ef236 [Kernel][DSv4.1] Bump FlashMLA to the 2026.09.30 release and its interleaved V4.1 KV cache (#61371) 
Signed-off-by: juntianl <juntianl@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-10-15 13:58:25 -07:00
Rohan PotdarandClaude Opus 5.5 105605b8e9 [ROCm] Support DeepSeek-V4 hash routing in AITER Triton W4A16 MoE (#60282) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 14:10:12 -05:00
Thien Tran 82b9ccaa79 [Perf][MoE] Skip the TRTLLM fp8-block activation-scale transpose copy (#61773) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-15 12:09:07 -07:00
Yongye ZhuClaude Opus 5.5Kevin H. Luu
 		4d58fa254a [MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 (#61538) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 19:02:01 +00:00
Mikko TukiainenandCursor 47fd76e4ce [ROCm][Perf] Route softmax MoE top-k, including fused shared-expert sigmoid, through AITER topk_gating (#56226) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-15 17:53:18 +00:00
Jhao-Ting ChenClaude Opus 5.5Robert Shaw
 		4f49b55c1f [MoE] Shape-based padding for FlashInfer TRT-LLM MXFP8 experts (gated and non-gated) (#61293) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-15 13:44:11 -04:00
MOVZX 76df77c04e Merge branch 'main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-15 21:16:39 +07:00
Tuukka SarviShanshan ShenKevin H. Luu
 		0951fc183c [ROCm][DSv4.1] Disambiguate ROCm attention class names (#58689) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 21:38:39 +08:00
Chaitanya Sri Krishna LollaandCursor 833665cded [ROCm][DI] Enable MoRI FP4 dispatch for mxfp4 w4a4 moe (#59596) 
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-14 23:20:14 -05:00
Tuukka SarviandKevin H. Luu 10d2c6083d [ROCm][DSv4.1] Enable DSpark adaptive verification (#58151) 
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 12:05:43 +08:00
Shanshan Shen f46021a7fa [ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 (#57773) 
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Signed-off-by: shen-shanshan <467638484@qq.com>
 2026-10-15 11:16:48 +08:00
Andreas KaratzasandClaude e96c822660 [Bugfix][ROCm][MiniMax-M3] Reject unbuilt index-head counts in the AITER indexer (#60990) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-14 14:13:48 -05:00
lucas-santos-amdandDouglas Lehr e534024e49 [ROCm][Bugfix][MLA] Fix FP8 MLA prefill unpad for interleave-padded heads (#60907) 
Signed-off-by: Lucas Santos <Lucas.Santos@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-14 14:34:28 -04:00
Kevin H. LuuandOpenAI Codex 0b6d07cbe2 [CI/Build] Use FileStore for DeepEP test rendezvous (#56595) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-14 14:29:12 -04:00
Zheng GongandCursor 355553ac22 [ROCm][Perf] Drop two dead per-step ops from the AITER sparse MLA metadata build (#58354) 
Signed-off-by: ZhengGong-amd <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-10-14 11:15:51 -05:00
Ashish RaoandVadim Gimpelson b4b7eca0ef [Bugfix][Quantization] Clamp FP8 scales before UE8M0 rounding (#58032) 
Signed-off-by: Ashish Rao <asrao@nvidia.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-10-14 13:56:42 +00:00
xuebwang 81503550f3 [ROCm] Disable AITER FP8/FP4 BMM for MLA by default (#53123) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-10-14 10:46:02 +00:00
Mikko TukiainenCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		f9b65c30c7 [ROCm][Perf] Make AITER FlyDSL GDN prefill default on gfx942/gfx950 (#60645) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-14 10:11:57 +00:00
Jiangyun ZhuandClaude Opus 5.5 badfcdbf91 [Perf][GDN] Internal prefill checkpoints for the FlashInfer GDN prefill backend (#60050) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 14:49:46 +08:00
Zheng GongCursorShanshan Shen
 		2308b6f00b [Perf][MoE] Cheaper triton_kernels 3.8 routing build in the OAI Triton MoE (#58897) 
Signed-off-by: Zheng Gong <zgong@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-10-14 10:48:05 +08:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ad8d39cbab [Bugfix] Fix DSpark graph-padding handling in the DeepSeek-V4.1 MoE fused top-k (#59166) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-14 07:49:52 +08:00
Anirudh Ravichandran e90158d912 [Bugfix][MoE] Clamp BLOCK_SIZE_K to the FP8 quantization block in the batched Triton MoE (#60733) 2026-10-13 17:02:17 -04:00
djramic 93d0bd9d50 [ROCm][CI] Drop the -inf padding check from the AITER MQA logits test (#61420) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-10-13 15:49:30 -04:00
Anish ShanbhagandOpenAI Codex 3f550129e8 [Perf][Attention][MiniMax-M3] Select CUTLASS MSA by EAGLE3 decode shape (#55838) 
Signed-off-by: Anish Shanbhag <ashanbhag@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-13 15:05:28 -04:00
Robert EsclapezClaude Opus 4.7Douglas Lehr
 		e83222396f [ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199) 
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-13 17:21:30 +00:00
Roberto L. CastroandClaude Opus 5.5 7fad7db330 [Perf][Feat] Add generic cuteDSL LL FP32 router (GEMM) (#44343) 
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 12:38:46 -04:00
Andrii SkliarAndrii SkliarLuke Alonso
 		6849ca696a [Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
 2026-10-13 09:06:33 -07:00
Elvir Crnčević b9b401b2ef [Kernel] Fuse DeepEP-v2 decode globalize+align into one launch (#53911) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
 2026-10-13 11:10:38 -04:00
simon-veitner-redhatLucas WilkinsonClaudeLucas WilkinsonMatthew Bonanni
 		31e90ac370 [Attention] Enable FA4 sparse MQA decode on Blackwell (#55866) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-10-13 09:31:37 -04:00
Vadim GimpelsonandClaude cd7517fb17 [Bugfix] Fix ReplaySSM ring shape assertion when TP does not divide Mamba2 n_groups (#60704) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-13 11:31:53 +00:00
JackRao123andClaude Opus 5.5 877f6df889 [Bugfix][DSv4] Don't bake the C128 compressor skip into CUDA graphs (#59709) 
Signed-off-by: Jack Rao <jack.rao@baseten.co>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 10:15:11 +00:00