iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,490 Commits 3 Branches 0 Tags
4a3ebc6b5bfcfa8cef4873f7ff213bf6bec4fef0
Commit Graph
1545 Commits
Author SHA1 Message Date
MOVZX 4a3ebc6b5b Merge branch 'main' into patches 2026-10-05 08:21:46 +00:00
Thien TranCodexnamgyu-youn
 		042ab0305c [Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections (#59533) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com>
 2026-10-05 07:03:29 +00:00
Gawr Gura 0721Gawr Gura 0721mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Wentao Ye
 		e8a53a55e6 [Bugfix][Attention] Support int4_per_token_head on non-power-of-two head sizes (#56198) 
Signed-off-by: Gawr Gura 0721 <LiRunGuo@users.noreply.github.com>
Co-authored-by: Gawr Gura 0721 <LiRunGuo@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-10-05 04:52:39 +00:00
MOVZX f83ecfac16 Merge remote-tracking branch 'upstream/main' into patches 2026-10-05 08:52:31 +07:00
bluemelovbluemelovCursorYifan QiaoClaude Opus 5.5
 		49e0f47978 [Bugfix][DSv4.1] Keep the compressor ring out of the null block (#58560) 
Signed-off-by: bluemelov <hannes.b2411@eva.space>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: bluemelov <hannes.b2411@eva.space>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 01:49:41 +00:00
00CC f98d1fc493 [ROCm][Bugfix] Fix ROCM_ATTN sliding-window boundary (#59550) 
Signed-off-by: tangzzycc <3081129260@qq.com>
 2026-10-04 13:48:46 +00:00
AIwork4meAIwork4meJartX
 		4ac0d0eac2 [ROCm][RDNA3] Fix W4A16 split-K accuracy and determinism (#54706) 
Signed-off-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
 2026-10-03 23:44:17 +08:00
MOVZX 85c31d47ad Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 22:06:10 +07:00
MOVZX 2877121b2d Merge remote-tracking branch 'upstream/main' into patches 
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-03 21:59:03 +07:00
Divakar Verma faa9860dbd [ROCm][CI][AiterExperts] Add test coverage for hidden/intermediate padding correctness (#59333) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-10-03 01:53:37 +00:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
Stefano CastagnettaandBenjamin Chislett f590eb2448 [Perf][Qwen4Exp] Add SM121 TP=1 skinny-GEMM plans (#59753) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-10-02 17:17:45 +00:00
Jinzhen LinCodexMisha Goin
 		9cf087fd14 [Perf] Tune MoE weighted-sum kernel launch configuration (#59731) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-02 17:09:29 +00:00
rmhaskarandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e42d35d4f3 [Minimax-M3] Keep the native FP8 MMA in the Triton indexer scorers (#59481) 
Signed-off-by: Rajas Mhaskar <rmhaskar@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 12:30:39 -04:00
Eric Tangaoshen02Claude Opus 5.5
 		4c4003bb5f [Bugfix] Bind routed-experts capture to the MoE layer, not the kernel (#59455) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 15:50:38 +00:00
DimensionSTPThien TranKimi Code
 		e4294144a8 [Perf] Keep non-speculative GDN decode on the standard path (#59735) 
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
 2026-10-02 13:44:53 +00:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 98b29aa99a [ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 11:47:41 +01:00
Simon Danielssonandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 751fa96a49 [ROCm][Perf][GLM-5.3-Flash] Remove redundant copy after ragged sparse MLA (#58569) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-02 11:45:51 +01:00
Yongye ZhuandClaude Opus 5.5 7566d83bd3 [Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path (#59300) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 14:39:55 -07:00
Jiangyun ZhuClaude Opus 5.5Thomas Parnell
 		cb6458c5ce [Core] Remove deprecated mamba_cache_mode "all" (#58997) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2026-10-01 02:59:37 -07:00
Ganesh RCursorDouglas Lehr
 		5addde2a0a [CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE (#59434) 
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-01 15:51:53 +08:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 15:18:25 +08:00
Wentao Ye 4d6874d137 [GLM 5.3 Perf] Skip qlnorm calculation for MHA, 4.4~7.7% E2E TTFT Improvement (#58845) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-01 15:13:57 +08:00
Juntian LiuandClaude Opus 5.5 4c2d277643 [Perf][DSv4.1] Faster Engram host lookups: sorted rows, inline big lookups (#59327) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 11:54:44 +08:00
namgyu-younandThien Tran 4ed438b79c [Perf][Qwen4Exp] Add SM100 low-latency decode GEMM plans (#59214) 
Signed-off-by: namgyu-youn <namgyu.dev@gmail.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-10-01 03:40:55 +00:00
Andrii SkliarandAndrii Skliar deb148e730 [Mamba] Add FlashInfer ReplaySSM support for MTP (#52928) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-09-30 21:41:17 +00:00
VenkyClaude Opus 5.5Zijing LiuKyle Liang
 		84f738f187 [Perf][MiniMax-M3] Triton indexer: decode grid retune + SM12.0 split-K (#56151) 
Signed-off-by: venkywonka <23023424+venkywonka@users.noreply.github.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Kyle Liang <199154931+kyleliang-nv@users.noreply.github.com>
 2026-09-30 23:33:23 +04:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Sergey Shlyapnikov 4e1182a3a6 [ROCm][MoE] Pad the AITER MoE intermediate size at allocation time, and round the expert-group count to a kernel that exists (#55368) 
Signed-off-by: Sergei Shliapnikov <sergei.shliapnikov@amd.com>
 2026-09-30 23:08:21 +08:00
almayneandLi, Jiang <jiang1.li@intel.com> 93dd80f636 [CPU] Conv1d optimised kernel for aarch64 (#54093) 
Signed-off-by: Anna Mayne <anna.mayne@arm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 20:23:42 +08:00
TaimysCodexLuyi XiaoTaimysHarry MellorClaude Opus 5.5
 		5225f1bacb [MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939) 
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 11:06:07 +00:00
Matej Sirovatkaaoshen02OpenAI Codexaoshen02
 		0103a9b96f [Quantization] Add per-token NVFP4 CuTe-DSL MoE backend (#50030) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-30 10:42:03 +00:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> 583637f126 [CPU][Perf] Add vectorized Sampler Kernel (#53913) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 15:16:28 +08:00
Fangzhou AiandCursor Agent 77e8b02c8d [ROCm][Perf] Add opt-in a4w4 (FP4 activation) MoE for DeepSeek V4.1 on AITER (#58819) 
Signed-off-by: fai <fangzhouai@gmail.com>
Signed-off-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
Co-authored-by: Cursor Agent (Claude Sonnet 5) <cursoragent@cursor.com>
 2026-09-30 15:07:44 +08:00
Harshal AdhavCursorLi, Jiang <jiang1.li@intel.com>
 		5faf81a429 [CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel (#58268) 
Signed-off-by: Harshal Adhav <harshal.adhav@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-30 13:52:53 +08:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 834ad456d3 [Refactor] Remove dead tests code (#58916) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 04:39:03 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
Chaitanya Sri Krishna Lollashikamd123Andreas Karatzas
 		467d81d9a0 [ROCm] Upgrade MoRI version on rocm dockers required for WideEP DP16 DI CI enablement and fixes for combine API (#56073) 
Signed-off-by: Shiksha Patel <shiksha.patel@amd.com>
Signed-off-by: lcskrishna <lollachaitanya@gmail.com>
Co-authored-by: shikamd123 <shikpate@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-29 18:07:31 +00:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		05d89636fc [ROCm][CI] Mirror generic GEMM-RS/AR on MI355 (#58433) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-29 10:28:15 +00:00
Andreas KaratzasandOpenAI Codex 69d5c81fcd [ROCm][CI] Mirror split kernel groups on MI355 and fix exposed tests (#58654) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Signed-off-by: Codex <codex@openai.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-29 02:08:31 -05:00
MOVZXandHinata ae0350c0de [Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863) 
Intake of open upstream PR vllm-project/vllm#58863. Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.

Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.

RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.

Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-09-29 13:39:09 +07:00
Mehmet CagriClaude Opus 5.5Shanshan Shen
 		63b9931b73 [ROCm][DSv4.1] Paged MXFP4 sparse indexer on aiter's MQA-logits kernel (#58671) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-29 14:37:57 +08:00
Muhammad Ahmed 0af34418e9 [ROCm][DSv4.1][Perf] Run the delayed mHC seams through aiter's fused Triton kernel (#58655) 
Signed-off-by: Ahmed <Muhammad.Ahmed@amd.com>
 2026-09-29 10:19:10 +08:00
Chaojun Zhang 7230dfea50 [XPU] Preserve non-contiguous strides when pinning CPU tensors for UVA view (#54874) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
 2026-09-29 09:43:07 +08:00
Mehmet CagriandClaude Opus 5.5 75fad5bbef [Bugfix][ROCm] Drop -1 sentinels when building the ragged sparse-MLA indices (#58058) 
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 18:40:11 -05:00
rmhaskarJarrel SeahTyler Michael SmithOpenAI Codex
 		e03875f4d2 [Minimax M3] Enable fp8 indexer cache on triton indexer for non-SM100 architectures. (#59081) 
Signed-off-by: Jarrel Seah <jarrel@harrison.ai>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Rajas Mhaskar <rmhaskar@nvidia.com>
Co-authored-by: Jarrel Seah <jarrel@harrison.ai>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 19:21:13 -04:00
stefankoncarevicandAndreas Karatzas 6c300ddb9e [CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP (#59008) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-28 17:29:59 +00:00
HDCharlesClaude Opus 4.6HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>OpenAI CodexOpenAI Codex
 		c01368fd64 [Quant] Use canonical N-first weight format for CT WNA16 MoE (#52798) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-28 11:22:44 -06:00
Thien Tran 39d49539de [KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel (#58651) 2026-09-27 18:18:06 -07:00
Fangzhou AiCursorAndreas Karatzas
 		31afa4f1bb [ROCm] Bump AITER to v0.1.23 (#58867) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-27 19:04:58 -05:00