iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
493 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Wentao Ye 8e25bbca19 [Refactor] Remove dead code multiple places (#61843) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-16 15:01:07 +00:00
Juan VillamizarJaden MathiasjpvillamOmar MuhammadDanielClaude Opus 5Daniel Chan
 		e711723352 [ROCm] gfx1250 optimizations and model enablement (#58800) 
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: Juan Villamizar <jpvillam@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: Omar Muhammad <omar.muhammad@amd.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
 2026-10-15 14:51:57 -07:00
Jhao-Ting ChenClaude Opus 5.5Robert Shaw
 		4f49b55c1f [MoE] Shape-based padding for FlashInfer TRT-LLM MXFP8 experts (gated and non-gated) (#61293) 
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-15 13:44:11 -04:00
Misha GoinClaudeKevin H. Luu
 		984c242ede [Quantization] Support compressed-tensors MXFP4 MoE with FP8 activations via Mxfp4MoEMethod (#59815) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-14 13:33:35 -07:00
Jinzhen LinCodexMisha GoinKevin H. Luu
 		fe98d90a9d [Dependency] Update Humming and remove NVFP4 scale workaround (#61556) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-14 13:32:36 -07:00
fxmarty-amdvllmellmTan Pin Siang
 		c9bfa574ce [Online quantization] Extend re-quantization API to MOE (showcase NVFP4 modelopt/CT/quark to any online MOE method) (#60276) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
 2026-10-14 15:49:37 +08:00
Stefano CastagnettaandKartikey Vyas 4e0681ed56 [Quantization] Support packed NVFP4 Qwen4Exp PLE embeddings (#56273) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai>
 2026-10-14 08:24:55 +08:00
MattClaude Opus 5.5Andreas Karatzas
 		e0317b28ce [Feature] Support batch invariance on ROCm (#52231) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-12 10:53:26 -05:00
fxmarty-amdOpenAI CodexMisha Goin
 		9f3d70e081 [Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-08 12:04:46 -04:00
stefankoncarevicandAndreas Karatzas 814301b268 [Bugfix][Quant] Read safetensors headers of checkpoints with other file names (#60411) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-08 01:38:41 -05:00
Xuanteng HuangClaude Fable 5.1OpenAI Codex
 		f38c6778ce [quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740) 
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-07 22:01:09 +00:00
Jinzhen LinandKevin H. Luu 32fbfa15e8 [Bugfix][Quantization] Accept folded NVFP4 scales for Humming A16 (#60337) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-07 11:04:43 -07:00
Tyler Michael SmithCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		5ab4e4445b [Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts (#59128) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 18:49:57 -05:00
Jinzhen LinCodexmgoinClaude Opus 5.5
 		d1f3d8b870 [Quantization] Prefer Humming before Marlin backends on SM90 (#56997) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 12:31:35 -07:00
lazymioClaude Fable 5.1Shengqi Chen
 		40c86ae1ce [Kernel] Enable OAI Triton MXFP4 MoE on SM12x (#58877) 
Signed-off-by: lazymio <mio@lazym.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-10-06 01:49:13 -04:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
pmanczakandPawel Manczak bdfb02674f [XPU][Kernel] Tune Triton W8A8 block-FP8 GEMM for Intel B70 (#56063) 
Signed-off-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
Co-authored-by: Pawel Manczak <203671809+pmanczak@users.noreply.github.com>
 2026-10-02 12:56:11 +01:00
Matej Sirovatkaaoshen02OpenAI Codexaoshen02
 		0103a9b96f [Quantization] Add per-token NVFP4 CuTe-DSL MoE backend (#50030) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-09-30 10:42:03 +00:00
xiao-llmandCursor 5840d95284 [Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071) 
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-25 20:51:18 -05:00
HDCharles 9f07d023d0 [Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors (#46528) 2026-09-23 10:23:11 -04:00
aoshen02andYongye Zhu 973a3be780 [Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions (#57732) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-23 01:16:20 -07:00
Matej Sirovatkaaoshen02OpenAI Codexaoshen02Claude Opus 5.5
 		9127170b88 [Quantization] Select per-token NVFP4 MoE backends explicitly (#57176) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-23 07:30:58 +00:00
Misha Goin 25965ee583 [Kernel] Remove AllSpark INT8 W8A16 GEMM backend (#58001) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-09-22 14:07:34 -07:00
Jinzhen LinCodexmgoin
 		7d99c2c4fd [Feature][Humming] Humming feature integration (#56685) 
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-20 08:11:01 -07:00
Misha GoinandErinYin e5fce7b56b [Core][Kernel] Share persistent workspaces for Marlin and Humming (#57421) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: ErinYin <liang.yin@daocloud.io>
 2026-09-20 17:28:33 +08:00
MattandClaude Opus 5 fef55fd26d [ROCm][CI] Adapt MoE tests to the triton_kernels 3.8 API (#57385) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-17 11:21:22 -05:00
Yuchen WangDakai AnOpenAI Codexaoshen02
 		78fdf4efb8 [Bugfix][MoE] Convert FlashInfer BF16 weights in place (#54699) 
Signed-off-by: Yuchen Wang <93700456+yuchenwang3@users.noreply.github.com>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-17 22:00:04 +08:00
Rohan PotdarClaude Opus 4.8Matt
 		3e267bae70 [ROCm] triton+triton_kernels 3.8 mxfp4 MoE support (gpt-oss + DeepSeek-V4) (#55934) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com>
 2026-09-16 20:03:34 -05:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Yongye ZhuJee Jee Liandyluo7OpenAI Codexzjy0516Claude Opus 5
 		e77daef89e [Model] Support DeepSeek-V4.1-Flash (#56214) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 02:11:19 -07:00
siyuandIsotr0py 73fb19151f [Fast Start] Support fp4 (#55465) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 13:22:17 +00:00
Andrii SkliarAndrii Skliarmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		83fe99399e [Spec Decode] Add NVFP4 DSpark gathered top-k projection (#55713) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-09 17:33:46 +00:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
jimmy-adams e41a17e606 [ROCm][Quantization] Support AMD Quark per-block FP8 for fused MoE layers (#52263) 
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
 2026-09-08 10:30:26 +00:00
Wei Zhao 6748217fb9 [Bugfix] Fix Kimi K3 NVFP4 MoE weight conversion OOM (#55407) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
 2026-09-07 11:45:25 +08:00
jimmy-adamsfxmarty-amdShanshan Shen
 		385ba6b5b0 [Bugfix][Quantization] Register Quark per-block FP8 scales as weight_scale (#54770) 
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-05 10:44:24 +08:00
Jiangyun Zhu 98ed0856f3 [Model] add GLM-5.3-Flash support (#53906) 2026-09-03 09:40:35 -07:00
fxmarty-amd 7894394b07 [Online quantization] Add targeted online quantization configuration based on user patterns (#51285) 2026-09-02 08:55:15 -04:00
Juhi MittalClaude Opus 4.8mgoin
 		7a977c0699 [ModelOpt] Redesign the LinearMethod classes using the generic QuantKey-driven method (#49381) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-01 16:28:56 -04:00
fxmarty-amdxuebwang-amdAndreas Karatzas
 		2fe5cef35e [Fix] Fix FSE compatibility detection for Quark-produced models (#54573) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: xuebwang-amd <xuebwang@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 12:12:20 -05:00
fxmarty-amdandAndreas Karatzas 63988f3c2d [Quantization][Refactor][1/N] Adopt QuantKey in QuarkConfig and methods, relying on weight_quant_key, act_quant_key for quant method dispatch (#52958) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 02:55:54 -05:00
+5
Harry Huangzjy0516KungYorkThien TranYifan Qiaoandyluo7Isotr0pyandyluo7OpenAI Codexlishunyang12Vadim GimpelsonIsotr0pyLucas WilkinsonClaude Opus 5OpenAI Codex
 		e126687a9a [Model] Support Qwen3.8-Flash-Next (#53896) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-31 05:57:56 +00:00
xuebwang-amd 06569a8696 [ROCm][Quantization][MOE] Enable fused shared experts for block-quantized FP8 (#53097) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-08-28 09:23:07 -07:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a5e0045433 [Refactor] Remove dead code quantization 2 (#52821) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-25 20:28:44 -07:00
Elvir CrnčevićClaude Opus 4.8Tyler Michael Smithmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ca29cd48ba [Quantization][Humming] Support MXFP4 weight + block-FP8 activation for MoE (#51332) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-25 10:21:00 -04:00
stefankoncarevic 7f4a1b7e24 [Bugfix][ROCM] Fix the MXFP8 block scale exponent (#53110) 
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-08-22 01:26:19 +00:00
vllm-agentandvllm-ci-failure-analyzer ba53da60bb Revert "[Bugfix][MoE] Tune FlashInfer experts to scheduler token limit" (#52989) (#53186) 
Co-authored-by: vllm-ci-failure-analyzer <kevin@inferact.ai>
 2026-08-21 12:39:28 -06:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00