 liangel-02
|
6b5c389ee3
|
expose flex block size for batch invariant mode (#41252)
Signed-off-by: Angel Li <liangel@meta.com>
|
2026-05-13 14:11:57 -07:00 |
|
|
Michael Goin
|
8efd508204
|
[Quantization] Rework quantization_config to use QuantKey and allow for activation override (#41566)
|
2026-05-13 16:58:32 -04:00 |
|
|
Wentao Ye
|
3f611f6106
|
[CI] Fix pre-commit issue (#42563)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-13 12:37:26 -07:00 |
|
|
fxmarty-amdandKyle Sayers
|
40330967ab
|
[Quark] Support loading Quark NVFP4 checkpoints in vLLM (#35859)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com>
|
2026-05-13 11:17:36 -07:00 |
|
|
TJianandYongye Zhu
|
a8887c208f
|
[Bugfix] [ROCm] [DSV4] [Perf] Add aiter mhc support (#41946)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-13 21:43:15 +08:00 |
|
|
PikaPikachu
|
3b1ef03be4
|
[Bugfix][Quark] Fix W8A8 INT8 garbage outputs on Step-3.5-Flash (and other 3-key fused-MoE Quark exports) (#41892)
Signed-off-by: kangletian <kangletian@hotmail.com>
|
2026-05-13 11:59:49 +00:00 |
|
|
bnellnm
|
18f6bf5a21
|
[MoE Refactor] Add sequence parallel tests to test_moe_layer.py (#41299)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-05-12 21:52:19 -04:00 |
|
|
Wentao Ye
|
3d635c58c0
|
[Perf] Optimize MLA compute_prefill_context memory allocation (#42460)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-12 16:23:46 -07:00 |
|
|
bnellnmandRobert Shaw
|
d9b4990783
|
[MoE Refactor] EPLB refactoring for FusedMoE (#41055)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-05-12 14:16:31 -04:00 |
|
|
|
4d591db470
|
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK (#40735)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
|
2026-05-12 13:37:44 -04:00 |
|
|
Jonathan MamouandWentao Ye
|
c8a6e272e0
|
[CPU] Fix rotary embedding for CPU without flash-attn ops (#42225)
Signed-off-by: jmamou <jonathan.mamou@intel.com>
Signed-off-by: Jonathan Mamou <jonathan.mamou@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-05-12 15:05:35 +00:00 |
|
|
bnellnm
|
6427603ae8
|
[MoE Refactor] Move remaining experts classes to experts directory (#42334)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-05-12 09:19:46 -04:00 |
|
|
|
206eaed08d
|
[MoE Refactor] Move expert map related code into ExpertMapManager class (#41046)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
|
2026-05-12 09:18:27 -04:00 |
|
|
Thomas ParnellandClaude Opus 4.6
|
8f89381fc6
|
[Hybrid] Warmup Mamba2 SSD kernel (#39822)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-05-12 12:46:22 +00:00 |
|
|
Dipika Sikka
|
a7b801e26d
|
[MXFP4] Support for linear layers + compressed-tensors integration (#41664)
|
2026-05-12 07:49:33 -04:00 |
|
|
amitz-nv
|
ef34592a1a
|
[Bugfix] Fix double reduce in flashinfer_nvlink_two_sided and flashinfer_nvlink_one_sided backends (#41382)
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
|
2026-05-12 07:47:47 +00:00 |
|
|
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bbee532988
|
[Perf][1/n] Eliminate various GPU<->CPU syncs (#41429)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 20:36:03 +00:00 |
|
|
Jee Jee LiandYongye Zhu
|
53181384e0
|
[Bugfix] Fix DSV4 swiglu_limit on marlin backend (#42287)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-11 13:03:56 -07:00 |
|
|
Vensenandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
6fdb49392e
|
[Bugfix] Fix int32 overflow in DeepGEMM SiLU/mul FP8 Triton kernel (#42201)
Signed-off-by: vensen <vensenmu@gmail.com>
Signed-off-by: Vensen <vensenmu@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-11 14:52:31 -04:00 |
|
|
Hexiang Wang
|
7863fff6e5
|
[ROCm][DSv4] implement flash sparse mla with triton kernels (#41812)
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
|
2026-05-11 09:27:11 -07:00 |
|
|
Wentao Ye
|
0d453e2336
|
[Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement (#40408)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-05-11 12:20:58 -04:00 |
|
|
Vadim Gimpelson
|
a2e776d716
|
[Bugfix] Accept canonicalized modelopt_* quant_method in _extract_modelopt_quant_algo (#42181)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
|
2026-05-11 11:10:57 -04:00 |
|
|
Wentao Ye
|
4b64fc2cbf
|
[Refactor] Cleanup batch invariant dead code (#41993)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-11 10:48:39 -04:00 |
|
|
Rohan PotdarandElizaWszola
|
a51376b3f0
|
[Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA (#40392)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
|
2026-05-11 14:10:50 +00:00 |
|
|
wangxiyuan
|
5536fc0c01
|
[Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
|
2026-05-11 03:59:36 +00:00 |
|
|
gnovackandgeorge
|
171019ab19
|
add fused mhc_post_pre kernel (#41536)
Signed-off-by: george <george@inferact.ai>
Co-authored-by: george <george@inferact.ai>
|
2026-05-10 19:56:52 -07:00 |
|
|
|
1b57eb41f2
|
[MoE] Move various experts classes to fused_moe/experts/ (#41979)
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
|
2026-05-11 07:54:33 +08:00 |
|
|
Vensenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
215e2f7990
|
[Bugfix][Mamba] IMA in causal_conv1d kernel for long sequences (#41617)
Signed-off-by: vensen <vensenmu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-10 12:38:28 +00:00 |
|
|
Isotr0py
|
48698b1b9b
|
[Bugfix] Fuse Qwen3.5 in_qkvz_proj forwarding with LoRA enabled (#37912)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-05-10 10:59:02 +00:00 |
|
|
Juhi Mittal
|
7a2b596982
|
[Quantization] Add ModelOpt NVFP4 W4A16 (4-bit weights, fp16/bf16 activations) support (#41769)
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
|
2026-05-09 21:15:50 +00:00 |
|
|
Thomas ParnellandJiangyun Zhu
|
3dda9aeb54
|
[Bugfix] Remove nested torch.compile in GDN rearrange_mixed_qkv causing CUDA graph capture failure (#42070)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-05-09 08:30:55 -07:00 |
|
|
Kermit
|
adb6d96516
|
[Bugfix] Fix GDN KKT precision loss on Hopper GPUs by aligning tl.dot operand layout with WGMMA (#42076)
Signed-off-by: kermit <ckeming@outlook.com>
|
2026-05-09 13:08:46 +00:00 |
|
|
Jee Jee Li
|
ecd0b60aad
|
[LoRA] Initial EP support for LoRA (#40867)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-05-09 00:31:23 -07:00 |
|
|
Terrence ZhaoandCursor
|
a2812becd6
|
[Models] Cohere Eagle + fix to Cohere MoE (#42078)
Signed-off-by: Terrencezzj <terrence@cohere.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-05-08 21:46:26 -07:00 |
|
|
Markus HartikainenandTJian
|
e8f9038ebd
|
[ROCm][Bugfix] Re-tag AITER MoE weights as preshuffled after replace_parameter (#42061)
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-05-08 21:42:07 -07:00 |
|
|
Michael Goin
|
8bcd8a260c
|
[Bugfix] Fix FlashInfer CUTLASS MXFP4-MXFP8 MoE by restoring swizzled scale (#42089)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-05-08 15:59:06 -07:00 |
|
|
|
2c6b59b807
|
[ROCm][Perf] Add Fused Shared Expert (FSE) support for Qwen3-Next (#39280)
Signed-off-by: nholmber <nholmber@users.noreply.github.com>
Signed-off-by: Tres Popp <tres.popp@amd.com>
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: nholmber <nholmber@users.noreply.github.com>
Co-authored-by: Tres <tpopp@users.noreply.github.com>
Co-authored-by: Tres Popp <tres.popp@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-05-08 15:38:00 -04:00 |
|
|
Hiroaki Mikami
|
90f145aaf7
|
[Models][Gemma3/Gemma4] Support hidden_act variants in gated MLP (#40588)
Signed-off-by: Hiroaki Mikami <hiroaki8270+github@gmail.com>
|
2026-05-08 11:29:11 -07:00 |
|
|
|
ed582b6a4c
|
[Aiter][ROCm] gdn_linear_attn kernel fusion (#40711)
Signed-off-by: Tres Popp <tres.popp@amd.com>
Signed-off-by: Chuan Li <chuali@amd.com>
Co-authored-by: hellozhuo <zhuo.su@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-05-07 23:11:37 -07:00 |
|
|
Stefano CastagnettaandYongye Zhu
|
54f548e9e5
|
[Bugfix] Restore moe_forward output shape invariant on TRTLLM MXFP4 path (#41646)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-07 15:26:06 -07:00 |
|
|
Kyle Sayers
|
c1819ca283
|
[Compressed Tensors] Allow configs with non-explicit ignores (#41965)
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
|
2026-05-07 14:03:45 -07:00 |
|
|
TomerBN-Nvidia
|
8189a15914
|
[Core] Replace routing replay with device cache and async D2H pipeline (#39917)
Signed-off-by: Tomer Barnatan <tbarnatan@nvidia.com>
|
2026-05-07 11:24:56 -07:00 |
|
|
Li, Jiang
|
b3945cc316
|
[CPU] Bump up to the latest CPU kernels (#41924)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-05-07 05:45:59 -07:00 |
|
|
Mehdi Ghanimifardandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d4b0048404
|
Eliminate redundant MoE buffer copies in AITER fused experts (without dependency on AITER changes) (#41713)
Signed-off-by: Mehdi Ghanimifard <mehdi.ghanimifard@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-07 03:46:40 -07:00 |
|
|
Yuwen Zhou
|
713b28bd0b
|
[CPU] Add FP8 W8A16 MoE support (#41314)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
|
2026-05-06 23:17:07 -07:00 |
|
|
Fadi ArafehandVadim Gimpelson
|
51f22dcfd0
|
[Feat][CPU] Enable Gated DeltaNet Attention (Qwen 3.5 / 3.6) (#41025)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
|
2026-05-07 12:57:09 +08:00 |
|
|
Yuankai Chenandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2e777d21a8
|
[Bugfix][Rocm]Aiter MoE re-uses existing tensor addresses after weight update. (#40390)
Signed-off-by: Yuankai Chen <yuankach@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-06 10:32:26 +00:00 |
|
|
Matthew Bonanni
|
01b9b5af67
|
[Attention] Minor refactor: layer takes ownership of the MLA prefill backend (#41744)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-05 23:22:41 +00:00 |
|
|
Matej SirovatkaandTyler Michael Smith
|
48954de237
|
Fix DeepGEMM ep_scatter output address overflow (#39213)
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2026-05-05 18:56:56 +00:00 |
|
|
|
628c436301
|
[New Model][ROCm] Add AMD support for DeepSeek V4 (#40871)
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com>
|
2026-05-05 08:55:37 -07:00 |
|