+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a5e0045433
[Refactor] Remove dead code quantization 2 ( #52821 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-25 20:28:44 -07:00
ba1cdcfcf0
[Model][Quantization] Add Ling-3.0-flash-fp8 support (#51265 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-09 23:09:08 -07:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
Julien Debache
ae934ba8a5
feat: extended EPLB support for Mistral Large 3 and additional MoE backends ( #48355 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-08-07 05:32:52 -07:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
7c77868cdf
[Bugfix] Size and iterate w13 by shard count for non-gated MoE ( #51125 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 17:24:47 -07:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Wentao Ye
0cf49a5d15
[Refactor] Remove multiple dead codes ( #50285 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-03 10:59:08 -04:00
Wentao Ye
b5bcb3ce88
[Refactor] Remove dead code in multiple files ( #49745 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-27 15:58:26 -04:00
Jinzhen Lin and mgoin
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
bnellnm and Robert Shaw
bc8481af09
[MoE Refactor] Standardize Humming MoE experts + utilities ( #43373 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-06-29 06:19:29 -07:00
Harry Mellor
5051698e41
Remove unnecessary load_weights methods ( #44589 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-29 01:52:23 -07:00
jj shao and shaojinjie.sjj
ea2ead1db3
[Misc] Fix incorrect layer type annotation in Fp8LinearMethod ( #46818 )
...
Signed-off-by: shaojinjie.sjj <shaojinjiesjj@gmail.com >
Co-authored-by: shaojinjie.sjj <shaojinjiesjj@gmail.com >
2026-06-27 20:23:59 +00:00
Yan Ma
acce57d8dd
Deprecate old FP8 online MoE quantization class ( #44514 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-06-23 11:53:38 -07:00
Wentao Ye
2659f60a1a
[Refactor] Remove dead quantization code and tests ( #45454 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-06-17 16:12:01 -04:00
Hongxia Yang and tjtanaa
e28e8c8782
[ROCm][Quant] Minimax-M3: Enable fp8_per_channel for bf16 weights on mi300x ( #45854 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-06-17 12:02:40 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3f53e2138f
[Refactor] Remove Fp8OnlineLinearMethod as scheduled ( #45463 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-16 04:35:58 -07:00
Giancarlo Delfin and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b5adb027ad
[Models] Fix MiMo v2.x QKV TP sharding + FP4 support ( #45200 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-15 15:13:34 +08:00
Michael Goin
6afa25000c
[Bugfix] Canonicalize FP8 weight layout to (K, N) at the source ( #44735 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-06-08 14:37:36 -06:00
Harry Mellor and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
62215e72c6
Remove KV cache scale boilerplate from model weight loading methods ( #43167 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-05 05:19:04 -07:00
Wentao Ye and Isotr0py
68e07d5916
[Bug] Fix ci issue assert output_size is not None AssertionError ( #43261 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-05-21 16:58:09 +08:00
Yongye Zhu and Claude
f8848b2f2d
[Bugfix] Add swiglu limits to deepgemm fp8 methods ( #41986 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-14 15:43:13 -07:00
4d591db470
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK ( #40735 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 13:37:44 -04:00
206eaed08d
[MoE Refactor] Move expert map related code into ExpertMapManager class ( #41046 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 09:18:27 -04:00
Wentao Ye
0d453e2336
[Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement ( #40408 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-05-11 12:20:58 -04:00
Markus Hartikainen and TJian
e8f9038ebd
[ROCm][Bugfix] Re-tag AITER MoE weights as preshuffled after replace_parameter ( #42061 )
...
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-05-08 21:42:07 -07:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
Yan Ma
595562651a
[XPU] fix MoE triton backend in online fp8 quantization ( #40109 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
2026-04-20 11:31:39 -04:00
Vasiliy Kuznetsov
38fa87caca
mxfp8 online quant move to new frontend ( #40152 )
...
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com >
2026-04-20 06:26:12 -07:00
vllmellm
5f7fab881a
[ROCm][FEAT] Integrate aiter gemm w8a8 ptpc ( #33773 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-04-16 09:55:29 +08:00
Michael Goin
11e2375fe2
[Refactor] Move MXFP8 GEMM management into MxFp8LinearKernel ( #39205 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-04-10 14:02:03 -07:00
Maral
2e9034c998
[W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. ( #33892 )
...
Signed-off-by: maral <maralbahari.98@gmail.com >
Signed-off-by: Maral <maralbahari.98@gmail.com >
2026-04-09 08:50:39 +08:00
Vasiliy Kuznetsov
7b1a7423be
[Frontend] new online quantization frontend ( #38138 )
...
Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com >
2026-04-03 11:58:39 -04:00
Qiming Zhang and Kunshang Ji
6b4872240f
[XPU] bump up xpu-kernel v0.1.5, transpose moe weights ( #38342 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-03 14:10:02 +00:00
bnellnm
7cf56a59a2
[MoE Refactor] Make SharedExperts class for use with DefaultMoERunner ( #35153 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-01 09:44:08 -04:00
Kyle Sayers
d28d86e8a3
[QeRL] Fix online quantized reloading ( #38442 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-29 14:56:41 -06:00
Kyle Sayers
648edcf729
[QeRL] Compose online quantization with quantized reloading ( #38032 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-03-27 13:22:33 -07:00
Bowen Bao
0ae89f18fd
[Refactor] Move FusedMoE hidden_size roundup to quant_method ( #34285 )
...
Signed-off-by: Bowen Bao <bowenbao@amd.com >
2026-03-26 23:38:26 -07:00
Vadim Gimpelson
52069012fe
[Bugfix] Fix DeepGemm E8M0 accuracy degradation for Qwen3.5 FP8 on Blackwell ( #38083 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
2026-03-26 01:21:47 -07:00
Ranran and Wentao Ye
dc6908ac6a
[Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning ( #35007 )
...
Signed-off-by: Ranran <1012869439@qq.com >
Signed-off-by: Ranran <hzz5361@psu.edu >
Signed-off-by: ran <hzz5361@psu.edu >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-03-23 18:31:14 -04:00
Kunshang Ji
27d5ee3e6f
[FP8]add FP8 WoQ kernel abstraction. ( #32929 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
2026-03-23 09:47:47 +00:00
97995f6376
[MoE Refactor] Create MK for TRTLLM Kernels ( #32564 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com >
2026-03-03 10:39:50 -08:00
6af03f2394
[Refactor] [1/N] Reorganize kernel abstraction directory ( #34055 )
...
Signed-off-by: BadrBasowid <badr.basowid@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-02-24 06:47:22 +00:00
Jia Guo and Claude Opus 4.6
ec85340531
[Quantization] Support FP8 MoE bias for models like GPT-OSS ( #34906 )
...
Signed-off-by: jasperjiaguo <jasperg662@gmail.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-02-23 19:07:47 -08:00
Asaf Joseph Gardin
1faa8cb73c
[Quantization] - Added uses_meta_device_weights to quant config ( #34645 )
...
Signed-off-by: Josephasafg <ajgard7@gmail.com >
2026-02-17 23:43:44 -08:00
Michael Goin
ff1f83b056
[Refactor] Replace activation: str with MoEActivation enum ( #33843 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
2026-02-11 17:29:32 -08:00
bnellnm
d1481ba783
[MoE Refactor] Introduce MoERunner abstraction and move execution logic from FusedMoE to DefaultMoERunner ( #32344 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-02-10 19:51:07 -05:00
TomerBN-Nvidia and Cursor
995bbf38f1
[Bugfix] Fix shared expert input for latent MoE in EP+DP (Nemotron-H) ( #34087 )
...
Signed-off-by: Tomer Natan <tbarnatan@nvidia.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-09 16:44:18 +00:00
zofia
9bdb06b436
[XPU][6/N] add xpu scaled_mm kernel ( #34117 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
2026-02-09 20:17:35 +08:00
