 sychen52
|
947138b6c2
|
Add nvfp4 kv cache support (#40177)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
|
2026-05-01 04:55:16 +00:00 |
|
|
Juhi Mittalandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
6b6ac6c3c7
|
[Kernel][MoE] Support GELU on TRT-LLM NvFP4 fused MoE for Gemma4 (#41050)
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-01 03:37:43 +00:00 |
|
|
Stefano Castagnetta
|
b542bdf7fb
|
[Bugfix] Disable FlashInfer CUTLASS MoE on SM110 (Jetson Thor AGX) (#40808)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
|
2026-04-30 20:08:49 -07:00 |
|
|
Woosuk Kwon
|
9c61864bf8
|
[DeepSeek] Use torch.mm for bf16xbf16->fp32 gemm (#41300)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-04-30 16:28:57 -07:00 |
|
|
Tran Le
|
71725f6730
|
[Bugfix] Fix RoutedExpertsCapturer for Gemma 4 MoE (top_k_experts) (#41401)
Signed-off-by: Tran Le <tranle@fireworks.ai>
|
2026-04-30 16:19:59 -07:00 |
|
|
|
b4806c8ee1
|
[DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided (#40960)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-30 15:33:12 -07:00 |
|
|
Michael Goin
|
75a4c166f2
|
Fix typo in log message for indexer cache (#41419)
Signed-off-by: Michael Goin <mgoin64@gmail.com>
|
2026-04-30 15:02:14 -07:00 |
|
|
fxmarty-amdandClaude
|
2917d6363a
|
[NVFP4][Hopper/AMD Instinct] Add Triton kernels for NVFP4 dequantization and QDQ emulation (#40033)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-04-30 17:35:48 -04:00 |
|
|
|
121dbe7a22
|
[ROCm] ROCm DeepEP API updated to latest (#39721)
Signed-off-by: Tej Kiran <vpolamre@amd.com>
Signed-off-by: tej <37236721+itej89@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: HAIAI <39548240+HAIAI@users.noreply.github.com>
|
2026-04-30 07:46:59 -07:00 |
|
|
|
b55b26520c
|
[MoE] Make MoERunnerInterface a PluggableLayer for OOT support (#35178)
Signed-off-by: wxsIcey <1790571317@qq.com>
Signed-off-by: Icey <1790571317@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-30 03:31:08 -07:00 |
|
|
Chenxi Qianandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
54146a9bf9
|
[Bugfix] correct h matrix layout in chunk_kda output kernel (#40956)
Signed-off-by: ChenxiQian <chenxi.qian.cq@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-30 16:22:41 +08:00 |
|
|
Rohan Potdar
|
3795d7acf4
|
[ROCm][Bugfix][GPTOSS]: fix input_ids and expert_map args for quark w4a8 gptoss (#41165)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
|
2026-04-29 16:39:01 -07:00 |
|
|
Hemanth Acharya
|
6841f5dc77
|
[ROCm] Add env flags to disable dynamic MXFP4 quant and enable AITER tuned GEMMs for Attention Projection Layers (#39987)
Signed-off-by: Hemanth Acharya <heachary@amd.com>
|
2026-04-29 16:07:46 -07:00 |
|
|
Rohan Potdar
|
944e138bcf
|
[ROCm][Bugfix]: W4A4 MOE using emulation instead of AITER on MXFP4-supported hardware (#41175)
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
|
2026-04-29 16:39:03 -05:00 |
|
|
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
faab189554
|
[Feature]: IndexCache support for DSA models (#37735)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-29 15:15:35 -04:00 |
|
|
Terrence Zhao
|
91a2d39014
|
[Models] Cohere MoE (#40817)
Signed-off-by: Terrencezzj <terrence@cohere.ai>
|
2026-04-29 15:54:54 +00:00 |
|
|
Wentao Ye
|
39a7f4f4e2
|
[Perf] Optimize AllPool.forward by slicing first, 51% faster in the method level benchmark (#41163)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-29 08:11:04 -07:00 |
|
|
Jee Jee Liandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
9d8ad5b408
|
[Bugfix] Fix repeated DSv4 RoPE cache initialization (#41148)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-29 20:29:55 +08:00 |
|
|
Lucas KabelaandWentao Ye
|
fa1b9840f6
|
[BE][Torch 2.12] Remove workaround code for fixed cublas issue (#40845)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: Lucas Kabela <lucasakabela@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-04-28 21:07:24 -07:00 |
|
|
chelnnexy
|
d109eacd05
|
[Bugfix][ROCm] Fix gemm_a4w4 call to use updated AITER API signature (#40754)
Signed-off-by: cheiluno <cheiluno@amd.com>
|
2026-04-29 09:04:53 +09:00 |
|
|
Yongye Zhu
|
5aa371dc8e
|
[DSV4] Enable Multi-stream for Pre-Attn GEMM (#41061)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-04-28 09:08:55 -07:00 |
|
|
zhangxin81
|
de3da0b97c
|
Add tuned triton fused_moe configs on H100 for gpt-oss (#39904)
Signed-off-by: zhangxin81 <115389973+zhangxin81@users.noreply.github.com>
|
2026-04-28 03:38:48 -07:00 |
|
|
anthonsu
|
76c9cccc36
|
[Core] Fix redundant None append in StepPool.forward for chunked prefill (#41049)
Signed-off-by: Anthony Su <xsuanthony@gmail.com>
|
2026-04-27 23:42:47 -07:00 |
|
|
JiangWeixiang
|
ed57f77192
|
[Bugfix ] fix bailing_moe_linear (#40859)
Signed-off-by: ghphotoframe <854746559@qq.com>
|
2026-04-27 22:39:23 -07:00 |
|
|
Andreas Karatzas
|
5e2c37facd
|
[ROCm][CI] Add missing quantization methods and fix online quant test failures (#39801)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-04-27 15:08:57 -05:00 |
|
|
|
c8bbe05189
|
[Perf] Update TRTLLM supported MoE routing methods (#39141)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: root <root@bia0030.bia.clusters.nvidia.com>
Co-authored-by: root <root@bia0036.bia.clusters.nvidia.com>
|
2026-04-27 14:16:22 -04:00 |
|
|
|
c245d35ff4
|
[Model] Add MiMo-V2.5 support (#40967)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <zhujiangyun@inferact.ai>
Co-authored-by: yasong <yasong.wang@inferact.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <copilot@github.com>
|
2026-04-27 13:26:51 +00:00 |
|
|
Xiaoshuang Wangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f8ac0c7cf0
|
[Bugfix] Fix k_norm weight sharding in MiniMaxM2Attention when total_num_kv_heads < tp_size (#38191)
Signed-off-by: wxsIcey <1790571317@qq.com>
Signed-off-by: Icey <1790571317@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-27 05:57:13 -07:00 |
|
|
Yongye Zhu
|
706a04d34b
|
[DSV4] Add silu clamp limit to shared expert (#40950)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-04-27 00:37:43 -07:00 |
|
|
Bhoomit
|
2cc008e7b4
|
[Attention][TurboQuant] Share dequant buffers, eliminate float16_copy (#40941)
Signed-off-by: Bhoomit Vasani <bhoomit.2010@gmail.com>
Signed-off-by: Vasani Bhoomit <bhoomit.2010@gmail.com>
|
2026-04-27 13:48:36 +08:00 |
|
|
Zhanda ZhuandYubo Gao
|
5d5c776444
|
[Perf] FP8 FlashInfer Attn for ViT (#38065)
Signed-off-by: Zhanda Zhu <zhandazhu@gmail.com>
Co-authored-by: Yubo Gao <ybgao-nvidia@users.noreply.github.com>
|
2026-04-27 13:44:15 +08:00 |
|
|
+7
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
|
Jee Jee Li
|
8cd174fa35
|
[LoRA] MoE LoRA Refactor (#40338)
|
2026-04-26 01:55:19 +00:00 |
|
|
Netanel HaberandJee Jee Li
|
12a3f6454b
|
[Bugfix][MoE] Only unpad routed output before shared expert add or routed output transform (#40865)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-04-25 20:50:12 +00:00 |
|
|
Wentao Ye
|
a474da2813
|
[Refactor] Remove unused dead code (#40640)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-25 07:28:18 +08:00 |
|
|
Artem Perevedentsev
|
333529deae
|
[EPLB] Fix replica selection bias in fused_moe router (#40810)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-04-24 22:06:41 +00:00 |
|
|
Jinzhen Lin
|
9f771b3ab9
|
[Quantization] add humming quantization kernel (#34556)
|
2026-04-24 09:29:44 -04:00 |
|
|
Itay Alroy
|
c9d3c6e6af
|
fused_moe: treat NIXL EP as batched experts (#40412)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-04-24 08:05:31 -05:00 |
|
|
Netanel Haber
|
e8eb0490ce
|
[Bugfix][MoE] Unpad routed output before shared expert add [Fixes #35949] (#40794)
Signed-off-by: Netanel Haber <nhaber@nvidia.com>
|
2026-04-24 11:53:23 +00:00 |
|
|
Jiangyun Zhu
|
e8ee2a78db
|
[Attention] use diff kv backend for mimo v2 flash (#40045)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-04-24 11:25:55 +00:00 |
|
|
Yuwen ZhouandKunshang Ji
|
4c34b2f6fc
|
[XPU] Enable torch.compile for XPU GDN attention (#39466)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-04-24 16:26:16 +08:00 |
|
|
Jackmin801andClaude
|
079a4cf399
|
[MoE] Move cutlass moe to fused_moe/experts/ (#40574)
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-04-24 06:05:49 +00:00 |
|
|
Nick Hill
|
fe85a92e86
|
[Core] Avoid seq_lens_cpu GPU->CPU sync (#40654)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-24 00:35:55 +00:00 |
|
|
Hemanth Acharya
|
fa4b70555b
|
[ROCm] Cast score correction bias tensor during model construction for DeepSeek/Kimi-K2 (#39999)
Signed-off-by: Hemanth Acharya <heachary@amd.com>
|
2026-04-24 09:02:12 +09:00 |
|
|
Jackmin801andRobert Shaw
|
447c372ac5
|
[MoE] Move remaining PrepareAndFinalize to prepare finalize folder (#39009)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
|
2026-04-23 20:00:53 -04:00 |
|
|
bnellnm
|
4a6dd1c3cc
|
[Bugfix] Fix DeepSeek V2-Lite Accuracy drop (#40673)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-04-23 18:11:37 -04:00 |
|
|
|
1b1c01de39
|
[MoE] Move xpu moe to fused_moe/experts/ (#40568)
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-04-23 13:38:10 -04:00 |
|
|
bnellnm
|
1c2c1eb8b9
|
[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-04-23 11:22:34 -04:00 |
|
|
|
b7a2605020
|
[Bugfix] Make Attention Backend Auto-Selection Batch-Invariance-Aware (#40193)
Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-23 14:57:03 +00:00 |
|
|
|
e4ee48da2d
|
[MoE refactor] refactor GPTQMarlinMoEMethod with MK (#37990)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
|
2026-04-23 05:21:47 +00:00 |
|