iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
228 Commits
Author SHA1 Message Date
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
afriedriSimon DanielssonKevin H. LuuAndreas Karatzas
 		2067297b2f [ROCm][Perf] Use AITER tuned configs to select a4w4 gemm kernel instead of env var (#61278) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 04:00:28 -05:00
Mikko Tukiainen 0c25849292 [ROCm][Perf][Bugfix] Probe the preshuffled FP4 GEMM config by logical K (#59406) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
 2026-10-15 21:11:33 +00:00
Ashish RaoandVadim Gimpelson b4b7eca0ef [Bugfix][Quantization] Clamp FP8 scales before UE8M0 rounding (#58032) 
Signed-off-by: Ashish Rao <asrao@nvidia.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-10-14 13:56:42 +00:00
xuebwang 81503550f3 [ROCm] Disable AITER FP8/FP4 BMM for MLA by default (#53123) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-10-14 10:46:02 +00:00
Robert EsclapezClaude Opus 4.7Douglas Lehr
 		e83222396f [ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches (#57199) 
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
 2026-10-13 17:21:30 +00:00
MattClaude Opus 5.5Andreas Karatzas
 		e0317b28ce [Feature] Support batch invariance on ROCm (#52231) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-12 10:53:26 -05:00
Jakub ZakrzewskiCodexMisha Goin
 		19c0bed584 [Fusion] Support 3D activation quantization fusion and enable it for Pixtral (#59612) 
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-09 15:12:03 +08:00
Kevin LiandMisha Goin fc494822ea [Test][Quantization] Gate CUTLASS W4A8 kernel tests to SM90 (#58878) 
Signed-off-by: Kevin Li <kevinbean99@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 19:14:35 +00:00
JartX 994b8b77be [Bugfix][ROCm] RDNA3 W4A16 MoE: handle the N-first CT weight layout (#60388) 
Signed-off-by: JartX <sagformas@epdcenter.es>
 2026-10-08 15:06:39 +00:00
00CC 5281e49908 [ROCm][Perf] Enable medium-skinny dispatch in RDNAHybridW4A16LinearKernel (#52619) 
Signed-off-by: tangzzycc <3081129260@qq.com>
 2026-10-07 13:21:43 +00:00
Jinzhen LinCodexmgoinClaude Opus 5.5
 		d1f3d8b870 [Quantization] Prefer Humming before Marlin backends on SM90 (#56997) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 12:31:35 -07:00
Matthias Gehre b267fe4881 [ROCm][Perf] W4A16: pad gfx11 weight and activation strides (#56301) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-10-06 08:22:33 -05:00
AIwork4meAIwork4meJartX
 		4ac0d0eac2 [ROCm][RDNA3] Fix W4A16 split-K accuracy and determinism (#54706) 
Signed-off-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: AIwork4me <AIwork4me@users.noreply.github.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
 2026-10-03 23:44:17 +08:00
Ganesh RCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e12291d733 [CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#54024) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Ganesh R <Ganesh.R@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 15:18:25 +08:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
HDCharlesClaude Opus 4.6HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>OpenAI CodexOpenAI Codex
 		c01368fd64 [Quant] Use canonical N-first weight format for CT WNA16 MoE (#52798) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: HDCharles <{"message":"Not Found","documentation_url":"https://docs.github.com/rest/users/emails#list-email-addresses-for-the-authenticated-user","status":"404"}>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-28 11:22:44 -06:00
MonishverClaude Fable 5.1Misha Goin
 		38cc054fc2 [Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization (#58194) 
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-25 09:19:48 -06:00
chuan932andMisha Goin f550ad8bf7 [Kernel][Perf] Register-resident path for per-token-group 8-bit quant (#55330) 
Signed-off-by: chao.huan <chao.huan@nio.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-09-25 08:11:42 -07:00
djramicandAndreas Karatzas 25b0add7b8 [ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-25 10:52:15 +00:00
mkunreddMahesh KunreddiClaudeAndreas KaratzasCodexAndreas Karatzas
 		4013f3ad6c [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 23:15:11 -05:00
Samuel Kimandmgoin f55419c43c [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) 
Signed-off-by: samuelkim7 <samuelmwkim@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-25 02:54:11 +00:00
Fangzhou AiandClaude Opus 5.5 3059155b47 [ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510) 
Signed-off-by: fai <fangzhouai@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-24 21:26:30 -05:00
Andreas KaratzasandOpenAI Codex 153242a314 [ROCm][CI] Add MI355 dense NVFP4 and MoRI kernel mirrors (#58281) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-23 03:06:15 -05:00
Misha Goin 25965ee583 [Kernel] Remove AllSpark INT8 W8A16 GEMM backend (#58001) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-22 14:07:34 -07:00
yinfengLiuandShanshan Shen 2af05512cf [Kernel] Add opt-in load-time MXFP4 dequantization (#50814) 
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-22 14:51:12 +08:00
afriedriDouglas LehrShanshan Shenmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		874a6b5a06 [ROCm] Refactor tuned gemms (#55001) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Signed-off-by: afriedri <afriedri@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 16:30:16 +00:00
Jinzhen LinCodexmgoin
 		7d99c2c4fd [Feature][Humming] Humming feature integration (#56685) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-20 08:11:01 -07:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
 2026-09-20 17:28:33 +08:00
Yuwen ZhouandLi, Jiang <jiang1.li@intel.com> b346479065 [CPU] Add CPU FP8 W8A8 linear/MoE support (#49942) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-09-18 17:59:42 +08:00
Thien TranandCodex 8da75d6190 C3x SM100 FP8 blockwise - Pad activation scales to multiple of 4 (#57377) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-18 08:39:27 +00:00
Misha Goin 9a5bd373cf [CI] Retire Weight Loading smoke tests (#57398) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-17 16:04:03 -04:00
afriedri a9a7e45f31 [ROCm] Restore VLLM_ROCM_USE_AITER_FP4_ASM_GEMM and default w4a4 ASM GEMM back to off (#57055) 
Signed-off-by: Andy Friedrich <afriedri@amd.com>
 2026-09-17 13:39:52 -05:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Li, Jiang b68408043d [Bugfix][CPU] Add per-tensor FP8 W8A16 kernel to fix Ministral crash on CPU (#56985) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-09-16 15:49:17 +08:00
Juntian LiuYongye ZhuClaude Opus 5Thien Tran
 		bb55077411 [Perf][Kernel] Integrate Mega-mHC from DeepGEMM for DeepSeek V4.1 (reopen of #56255) (#56962) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-09-15 04:01:06 -07:00
Netanel Haber d4ee7fe7a9 [Quantization] Select linear backends per quantization (#51204) 
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2026-09-14 16:24:18 -04:00
Matthias Gehre dc2e8f1157 [ROCm][Perf] W4A16: keep skinny GEMM zero-points packed 4-bit (#54965) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
 2026-09-14 17:40:25 +08:00
Ayushman SinghandAyushman Singh c2ea9f1a5f [Kernel][Perf][Quantization] Fix odd-row performance cliff in per-token-group quantization (#56478) 
Signed-off-by: Ayushman Singh <ayush1399@users.noreply.github.com>
Co-authored-by: Ayushman Singh <ayush1399@users.noreply.github.com>
 2026-09-12 20:27:08 -07:00
JartX 986e2f870a [Refactor][ROCm] Migrate the RDNA3 W4A16 MoE to the oracle/experts pa… (#55522) 
Signed-off-by: JartX <sagformas@epdcenter.es>
 2026-09-12 21:20:26 +08:00
Yongye ZhuJee Jee Liandyluo7OpenAI Codexzjy0516Claude Opus 5
 		e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 02:11:19 -07:00
pmanczak 6376c601ed [XPU][Tests] Enable test_per_token_group_quant_int8 on XPU (#55681) 
Signed-off-by: pmanczak <pawel.manczak@intel.com>
 2026-09-11 15:59:13 +08:00
MattandClaude Opus 5 9163190dda [ROCm][Bugfix][Perf] Tune multi-stream shared experts use; wvSplitKrc fixes (#56098) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-10 15:14:22 -07:00
Yongye ZhuandClaude Opus 5 be1cb9834b [Kernel] Optional Q-norm in fused DSv4 MLA epilogue; group_size=32 for packed FP8 quant (#56215) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-10 00:35:59 -07:00
Colin Z 588a813a60 [ROCm] [BugFix] Fix AITER MXFP4 ASM-GEMM crash on unfused shared experts (#55213) 
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com>
 2026-09-10 05:23:36 +00:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
Raya Elena Solanoandmgoin bcca76e7db [Test] Dequantize NVFP4 KV cache scales in the layout the kernel writes (#55908) 
Signed-off-by: Raya Elena Solano <raya.solano@mbinf.de>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 17:13:39 +00:00