fxmarty-amd
7894394b07
[Online quantization] Add targeted online quantization configuration based on user patterns ( #51285 )
2026-09-02 08:55:15 -04:00
Micah Williamson
ce6a283c09
[Bugfix] Restore weight_dtype in QuarkW8A8Fp8MoEMethod to fix GPT-OSS FP8 MoE weight loading ( #54824 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-09-01 16:08:11 -05:00
fxmarty-amd and Andreas Karatzas
63988f3c2d
[Quantization][Refactor][1/N] Adopt QuantKey in QuarkConfig and methods, relying on weight_quant_key, act_quant_key for quant method dispatch ( #52958 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-01 02:55:54 -05:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a5e0045433
[Refactor] Remove dead code quantization 2 ( #52821 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-25 20:28:44 -07:00
Wentao Ye
29c9af5211
[Kimi K3 Refactor] Add UnfinalizedMoEOutput proto following up for #53152 ( #53310 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-24 11:21:11 -04:00
ErenAta16
fbb17e780b
[Bugfix] Fix six quantization exception messages split across positional args ( #50479 )
...
Signed-off-by: ErenAta16 <erena6466@gmail.com >
2026-08-21 09:06:56 -07:00
Luke Alonso and mgoin
2740c817ff
[Kernel] Add b12x FP4 MoE backend ( #52018 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-08-21 08:04:36 -07:00
fxmarty-amd
88b2bff2c6
[MOE] Standardize and abstract fused shared expert optimization selection ( #51695 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-18 12:56:56 -04:00
fxmarty-amd
48825acc23
[Quantization] Remove dead QuantizationConfig.is_mxfp4_quant ( #51793 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-13 13:58:31 -04:00
Rohan Potdar and Claude Opus 4.8
0f0cb918b7
[ROCm][MoE] Fix expert_map vs AITER expert_mask for non-AITER experts under EP ( #49758 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-11 15:33:02 -05:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
da788334bc
Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel ( #47972 )
...
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: huji <huidong.ji@amd.com >
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: huji <huidong.ji@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-07 01:19:31 -07:00
e08111211b
[Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation ( #47106 )
...
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-06 17:42:45 -07:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
7c77868cdf
[Bugfix] Size and iterate w13 by shard count for non-gated MoE ( #51125 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 17:24:47 -07:00
fxmarty-amd
e67a2e0a56
[ROCm][Quark][7/N] Use MXFP4 linear kernel abstraction for emulation backend ( #48949 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-07-31 10:07:15 -07:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
fxmarty-amd and Andreas Karatzas
7aea73d83d
[ROCm][Quark][6/N] Use MXFP4 linear kernel abstraction for aiter backend ( #49348 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-27 21:35:57 -05:00
1053e248f0
[ROCm][Quantization][5/N] Refactor quark_moe w8a8-int8 w/ oracle ( #46765 )
...
Signed-off-by: amd-sourjya <amd-sourjya@users.noreply.github.com >
Co-authored-by: amd-sourjya <amd-sourjya@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-27 16:01:34 -05:00
xuebwang-amd
f12b80c6ef
[ROCm][Bugfix] Fix GPT-OSS Quark MXFP4 MoE loading - emulation buffer not block-aligned ( #43979 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
2026-07-18 03:49:41 +00:00
xiao-llm and Andreas Karatzas
02bf9c7907
Fix Quark mxfp4 quantized model loading issue under mtp ( #46757 )
...
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-16 14:58:47 -04:00
music-dino
7d56fe2adc
[ROCm][CI] Avoid HIP init at config time via lazy aiter import in Quark OCP-MX ( #48015 )
...
Signed-off-by: Dino Music <Dino.Music@amd.com >
2026-07-16 16:47:08 +00:00
Jinzhen Lin and mgoin
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
576bf75d0e
[AMD][EPLB] Enable EPLB for Quark OCP MXFP4 MoE ( #47220 )
...
Signed-off-by: okorzh-amd <okorzh-amd@users.noreply.github.com >
Co-authored-by: okorzh-amd <okorzh-amd@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-03 14:41:52 -05:00
bnellnm and Robert Shaw
bc8481af09
[MoE Refactor] Standardize Humming MoE experts + utilities ( #43373 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-06-29 06:19:29 -07:00
Harry Mellor
5051698e41
Remove unnecessary load_weights methods ( #44589 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-29 01:52:23 -07:00
fxmarty-amd
e368415daa
[AMD][OCP MX][CI] Fix tests to not dispatch on UNFUSED_TRITON backend on MI300, improve w_mxfp4_a_fp8 emulation support ( #46142 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-06-23 14:25:27 -04:00
Bowen Bao and Andreas Karatzas
c97e8f99d6
[ROCm][Quantization][4/N] refactor quark_moe fp8 w/ oracle ( #43721 )
...
Signed-off-by: Bowen Bao <bowenbao@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-06-22 15:58:03 -07:00
qli88
d112eb1ac7
[feature] MiniMax-M3-MXFP4 support added ( #45896 )
...
Signed-off-by: Qiang Li <qiang.li2@amd.com >
2026-06-17 18:50:48 +00:00
xaguilar-amd
0b131b16c9
[ROCm][AITER][Quark] Tag per-channel FP8 weights as PER_CHANNEL so AITER pre-shuffled GEMM is selected ( #44626 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
2026-06-17 14:05:34 +00:00
Shantipriya Parida
a1ec011a83
[Bugfix] Add deepseek_v32 to Quark dynamic MXFP4 model type check ( #39498 )
...
Signed-off-by: Shantipriya Parida <shantipriya.parida@amd.com >
2026-06-10 02:52:33 -07:00
Harry Mellor
a80af24356
Speed up docs build ( #44635 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-05 14:51:44 +00:00
Harry Mellor and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
62215e72c6
Remove KV cache scale boilerplate from model weight loading methods ( #43167 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-05 05:19:04 -07:00
Yongye Zhu
2d2c660104
[MoE] Remove inplace fused experts mechanism ( #43727 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-05-27 20:00:19 -07:00
Bowen Bao
a2c8fc6657
[ROCm][Quantization][3/N] Refactor quark_moe w4a4 w/ oracle ( #41436 )
...
Signed-off-by: Bowen Bao <bowenbao@amd.com >
2026-05-18 13:46:13 -04:00
Yongye Zhu and Claude
f8848b2f2d
[Bugfix] Add swiglu limits to deepgemm fp8 methods ( #41986 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-05-14 15:43:13 -07:00
Wentao Ye
3f611f6106
[CI] Fix pre-commit issue ( #42563 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-05-13 12:37:26 -07:00
fxmarty-amd and Kyle Sayers
40330967ab
[Quark] Support loading Quark NVFP4 checkpoints in vLLM ( #35859 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: fxmarty-amd <felmarty@amd.com >
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com >
2026-05-13 11:17:36 -07:00
PikaPikachu
3b1ef03be4
[Bugfix][Quark] Fix W8A8 INT8 garbage outputs on Step-3.5-Flash (and other 3-key fused-MoE Quark exports) ( #41892 )
...
Signed-off-by: kangletian <kangletian@hotmail.com >
2026-05-13 11:59:49 +00:00
4d591db470
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK ( #40735 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 13:37:44 -04:00
206eaed08d
[MoE Refactor] Move expert map related code into ExpertMapManager class ( #41046 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 09:18:27 -04:00
1b57eb41f2
[MoE] Move various experts classes to fused_moe/experts/ ( #41979 )
...
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com >
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com >
2026-05-11 07:54:33 +08:00
Bowen Bao
1e9500410a
[ROCm][Quantization][2/N] Refactor quark_moe w4a8 w/ oracle ( #39136 )
...
Signed-off-by: Bowen Bao <bowenbao@amd.com >
2026-05-04 19:50:38 -07:00
Rohan Potdar
3795d7acf4
[ROCm][Bugfix][GPTOSS]: fix input_ids and expert_map args for quark w4a8 gptoss ( #41165 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
2026-04-29 16:39:01 -07:00
Hemanth Acharya
6841f5dc77
[ROCm] Add env flags to disable dynamic MXFP4 quant and enable AITER tuned GEMMs for Attention Projection Layers ( #39987 )
...
Signed-off-by: Hemanth Acharya <heachary@amd.com >
2026-04-29 16:07:46 -07:00
Rohan Potdar
944e138bcf
[ROCm][Bugfix]: W4A4 MOE using emulation instead of AITER on MXFP4-supported hardware ( #41175 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
2026-04-29 16:39:03 -05:00
chelnnexy
d109eacd05
[Bugfix][ROCm] Fix gemm_a4w4 call to use updated AITER API signature ( #40754 )
...
Signed-off-by: cheiluno <cheiluno@amd.com >
2026-04-29 09:04:53 +09:00
Andreas Karatzas
5e2c37facd
[ROCm][CI] Add missing quantization methods and fix online quant test failures ( #39801 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-27 15:08:57 -05:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
fxmarty-amd