54dd98be28
[CT] Support Humming for WNA16 MoE ( #48918 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-19 12:31:32 -04:00
Andreas Karatzas
160f7f0840
[ROCm][CI] Extended Fused MoE and FP8 MoE test support ( #41100 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-19 10:19:42 -05:00
Yohann Prigent and Misha Goin
be06873198
[Bugfix] compressed-tensors: restore int8 grouped WNA16 MoE support ( #52002 )
...
Signed-off-by: Yohann Prigent <prigent.yohann@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-08-19 07:55:59 -07:00
fxmarty-amd
88b2bff2c6
[MOE] Standardize and abstract fused shared expert optimization selection ( #51695 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-18 12:56:56 -04:00
qli88
e8ad2855e7
[Bugfix][ROCm] Fix a few int4/int8 quantization errors ( #52112 )
...
Signed-off-by: Qiang Li <qiang.li2@amd.com >
2026-08-18 08:31:51 +00:00
69d3335066
[Bugfix][Quantization] Guard the MXFP8 FlashInfer path on FlashInfer availability ( #52648 )
...
Signed-off-by: Han Li <han.li124@imperial.ac.uk >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-08-17 18:38:42 -07:00
Kunshang Ji
0e8989b416
[ROCm] gaurd on_gfx1250 call with rocm platform ( #52625 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-18 08:05:48 +08:00
Wentao Ye
7d7b6f26f4
[Refactor] Remove dead code for quantization ( #52221 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-16 13:10:16 -04:00
sroberts-amd and Stacy Roberts
c794754062
[ROCm]Remove special-case SiTU support model-specific gating ( #50597 )
...
Signed-off-by: Stacy Roberts <sroberts@amd.com >
Signed-off-by: Stacy Roberts <robers23@utexas.edu >
Co-authored-by: Stacy Roberts <robers23@utexas.edu >
2026-08-14 13:38:04 -05:00
fxmarty-amd
48825acc23
[Quantization] Remove dead QuantizationConfig.is_mxfp4_quant ( #51793 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-13 13:58:31 -04:00
19d61b1273
[Bugfix] Initialize DeepGemmQuantScaleFMT oracle lazily; bound QuantFP8 UE8M0 packed path to group_size 128 ( #51359 )
...
Signed-off-by: BabyDrangoner <148877251+BabyDrangoner@users.noreply.github.com >
Signed-off-by: kiroxu <148877251+BabyDrangoner@users.noreply.github.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-12 14:27:50 -04:00
Tan Pin Siang
e60f3c4c13
[Bugfix] Fix MiniMax-M3 compressed-tensors FP8 MoE SwiGLU params ( #46845 )
...
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com >
2026-08-12 10:52:30 +00:00
Chaojun Zhang
8c011da6d0
[XPU] Route block-quantized FP8 weights to the W8A8 kernel ( #50787 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-08-12 16:57:22 +08:00
Matej Sirovatka and Jee Jee Li
3ac9525507
[Bugfix][Quantization] Reuse online NVFP4 MoE kernel across reloads ( #50074 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-12 14:06:53 +08:00
Rohan Potdar and Claude Opus 4.8
0f0cb918b7
[ROCm][MoE] Fix expert_map vs AITER expert_mask for non-AITER experts under EP ( #49758 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-11 15:33:02 -05:00
ba1cdcfcf0
[Model][Quantization] Add Ling-3.0-flash-fp8 support (#51265 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-09 23:09:08 -07:00
Isotr0py
3b4c86e489
[Bugfix][Quantization] Fix fp32 weight scale for mxfp4 quantization and per-expert checkpoint mapping ( #51419 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-09 23:08:52 -07:00
Lena Onyshchenko and Kunshang Ji
751f2ccdd3
[Kernel][XPU] Tensor-descriptor operand loads for Triton W8A8 scaled_mm ( #47205 )
...
Signed-off-by: oonyshch <xonyshch@gmail.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-10 00:56:11 +00:00
Isotr0py
073c510c91
[Migration] Migrate bitsandbytes support to OOT plugin ( #43529 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-09 10:27:31 +08:00
Nick Hill
9823714346
[Bugfix] Drop stale layer kwarg from online MXFP4 kernel creation (fix precommit) ( #51442 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-07 13:49:00 -07:00
fxmarty-amd
3518110f2f
[Online quantization] Add online MXFP4 quantization support ( #49347 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-07 12:39:46 -07:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
Xiaochang Wu and Kunshang Ji
d5aae2b464
Fix ROCm architecture import on non-ROCm platforms ( #51357 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-07 05:32:56 -07:00
Julien Debache
ae934ba8a5
feat: extended EPLB support for Mistral Large 3 and additional MoE backends ( #48355 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-08-07 05:32:52 -07:00
da788334bc
Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel ( #47972 )
...
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: huji <huidong.ji@amd.com >
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: huji <huidong.ji@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-07 01:19:31 -07:00
e08111211b
[Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation ( #47106 )
...
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-06 17:42:45 -07:00
8170c23c4f
[Quantization] Share online weight scales across TP ( #49764 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-06 16:34:33 -07:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
Matej Sirovatka
9c22668436
[Quantization] Preserve precision in online NVFP4 expert packing ( #50029 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
2026-08-06 02:23:54 -07:00
b976e980ef
[XPU] Route AWQ linear through choose_mp_linear_kernel ( #50840 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <noreply@opencode.ai >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:50:11 +08:00
7c77868cdf
[Bugfix] Size and iterate w13 by shard count for non-gated MoE ( #51125 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 17:24:47 -07:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Netanel Haber
482c613423
[Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions ( #51093 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-08-05 14:08:55 -07:00
Yejing Lai and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2cb3ff88cc
[BUGFIX][Quant]Fix test_kv_scale_reload failed ( #50405 )
...
Signed-off-by: Lai, Yejing <yejing.lai@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 10:50:54 +08:00
0b37d8389f
fix: NVFP4 quantization out_dtype should match model dtype, not torch default ( #48861 )
...
Signed-off-by: Chris Fontes <2224082+fattchris@users.noreply.github.com >
Co-authored-by: Chris Fontes <chris@fontes.io >
Co-authored-by: Chris Fontes <chris@fontes.dev >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-03 22:47:37 +00:00
Hongxia Yang
8f50685c48
[ROCm][Kimi-K3] aiter moe environment variable cleanup ( #50582 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
2026-08-03 18:54:04 +00:00
wangxiyuan
dd11df04f3
[Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation ( #49389 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-08-03 11:15:13 -04:00
Wentao Ye
0cf49a5d15
[Refactor] Remove multiple dead codes ( #50285 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-03 10:59:08 -04:00
zofia and Kunshang Ji
e481da9508
[XPU] [Linear] add torch as xpu linear backend ( #49664 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-03 14:30:21 +08:00
Qiming Zhang
2755489a26
[INC] fix w4a4 model ( #50807 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-08-03 14:28:58 +08:00
e3be89673d
Enable ModelOpt FP8 emulation on SM80 ( #50019 )
...
Signed-off-by: Mike G <180722391+mikekg@users.noreply.github.com >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 14:48:27 -07:00
454ea5b526
[MoE Refactor] Combine CompressedTensorsWNA16MarlinMoEMethod with CompressedTensorsWNA16MoEMethod ( #44570 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Signed-off-by: Dobrzyniewicz, Agata <agata.dobrzyniewicz@intel.com >
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com >
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyn@users.noreply.github.com >
Co-authored-by: Tushar Jain <8455015+tushar00jain@users.noreply.github.com >
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-07-31 14:13:03 -07:00
fxmarty-amd
e67a2e0a56
[ROCm][Quark][7/N] Use MXFP4 linear kernel abstraction for emulation backend ( #48949 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-07-31 10:07:15 -07:00
Andreas Karatzas
5233368dac
[ROCm][CI] Fall back to lossless Kimi K3 MXFP4 emulation on gfx942 ( #50516 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-07-31 11:33:16 -04:00
bnellnm and Harry Mellor
6e311c6e20
[MoE Refactor] Rename FusedMoE to FusedMoEFactory ( #44941 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-31 11:28:18 +01:00
Fadi Arafeh
0e9b50074e
[chore] clean-up weight prepack for INT8 MoE ( #50116 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-07-31 10:15:38 +00:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
Ryan Clark and Claude Fable 5
68fb303a41
[Bugfix] Preserve Marlin runtime tensor storage across weight reload ( #48438 )
...
Signed-off-by: Ryan Clark <ryanclark2k@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-30 14:50:03 -07:00
Netanel Haber and OpenAI Codex
70bd10930b
[Quantization] Honor --linear-backend for ModelOpt W4A16 ( #50273 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 13:19:56 -07:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
