 Simon DanielssonandAndreas Karatzas
|
3d204dfdaa
|
Revert "[Perf][ROCm] Dual-stream decode with hipgraphs" (#52024)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-13 04:38:34 +00:00 |
|
Simon Danielsson
|
47ececb58e
|
[Perf][ROCm] Dual-stream decode with hipgraphs (#48223)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
|
2026-08-12 09:32:24 +08:00 |
|
Wentao Ye
|
fd04edef3e
|
[Refactor] Delete dead code in models (#51838)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-11 15:44:26 -04:00 |
|
Julien Debache
|
52be12cfac
|
feat: allow shared expert overlapping for FlashInfer one-sided all-to-all (#50569)
Signed-off-by: jdebache <jdebache@nvidia.com>
|
2026-08-11 10:06:22 -07:00 |
|
bnellnm
|
373fe8b83e
|
[MoE Refactor] Remove MoE legacy code (#51078)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-08-05 14:35:36 -07:00 |
|
Wentao Ye
|
d31de3c421
|
[Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved (#50912)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-04 14:00:44 -04:00 |
|
Jee Jee Li
|
b9d1e2437e
|
K3: Move LatentMoERunner (#50678)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-08-03 09:36:17 +00:00 |
|
Jee Jee Li
|
4635cc3e8f
|
Shard the K3 Latent-MoE up-projection on large batches (#50383)
|
2026-08-03 00:49:39 -07:00 |
|
 bnellnmandHarry Mellor
|
6e311c6e20
|
[MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-31 11:28:18 +01:00 |
|
+13        
|
aeeb36b1f1
|
[New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-07-30 03:49:58 -07:00 |
|
+12        
|
7c6729b769
|
[Model] Add Kimi K3 support: model files and kernels [1/N] (#50089)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-07-29 14:10:58 +08:00 |
|
  
|
b6ff8a2f50
|
[Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-19 02:53:15 +00:00 |
|
 Wei ZhaoandYongye Zhu
|
2afa3f7e95
|
[Perf] Minimax M3 - Support cross-layer allreduce-norm fusion (#47631)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-07-07 21:16:32 -07:00 |
|
 Wentao YeandCanlin Guo
|
b6cc46ec3b
|
[Feature] Support sequence parallel without the need for DP, 1.9%~5.0% E2E Throughput Improvement (#47070)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
|
2026-07-05 05:41:30 -07:00 |
|
Harry Mellor
|
1ab9522935
|
Remove more unnecessary load_weights methods (#47058)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-30 15:22:16 +01:00 |
|
Woosuk Kwon
|
89876b0c54
|
[GLM5] Implement op fusion for GLM5/DSV3.2 (#46876)
|
2026-06-28 08:17:39 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Netanel Haberandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c5e5c33fcd
|
[Bugfix][MoE] Restore routed output unpadding before shared expert add (#45707)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-16 16:06:28 +03:00 |
|
bnellnm
|
dc68bd8c41
|
[MoE Refactor] FusedMoE/MoERunner inversion refactor (#41184)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-06-08 10:42:58 -04:00 |
|
Yongye Zhu
|
2d2c660104
|
[MoE] Remove inplace fused experts mechanism (#43727)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-27 20:00:19 -07:00 |
|
Jee Jee Li
|
ec5de7fa7d
|
[LoRA] Add one shot triton kernel For MoE LoRA (#42290)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-05-25 19:47:04 -07:00 |
|
aoshen02
|
8c79ad6580
|
Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
2026-05-13 23:49:01 -07:00 |
|
 
|
4d591db470
|
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK (#40735)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
|
2026-05-12 13:37:44 -04:00 |
|
     
|
2c6b59b807
|
[ROCm][Perf] Add Fused Shared Expert (FSE) support for Qwen3-Next (#39280)
Signed-off-by: nholmber <nholmber@users.noreply.github.com>
Signed-off-by: Tres Popp <tres.popp@amd.com>
Signed-off-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: nholmber <nholmber@users.noreply.github.com>
Co-authored-by: Tres <tpopp@users.noreply.github.com>
Co-authored-by: Tres Popp <tres.popp@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
|
2026-05-08 15:38:00 -04:00 |
|
 Stefano CastagnettaandYongye Zhu
|
54f548e9e5
|
[Bugfix] Restore moe_forward output shape invariant on TRTLLM MXFP4 path (#41646)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-07 15:26:06 -07:00 |
|
TomerBN-Nvidia
|
8189a15914
|
[Core] Replace routing replay with device cache and async D2H pipeline (#39917)
Signed-off-by: Tomer Barnatan <tbarnatan@nvidia.com>
|
2026-05-07 11:24:56 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
b55b26520c
|
[MoE] Make MoERunnerInterface a PluggableLayer for OOT support (#35178)
Signed-off-by: wxsIcey <1790571317@qq.com>
Signed-off-by: Icey <1790571317@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-04-30 03:31:08 -07:00 |
|
+7        
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
 Netanel HaberandJee Jee Li
|
12a3f6454b
|
[Bugfix][MoE] Only unpad routed output before shared expert add or routed output transform (#40865)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-04-25 20:50:12 +00:00 |
|
Netanel Haber
|
e8eb0490ce
|
[Bugfix][MoE] Unpad routed output before shared expert add [Fixes #35949] (#40794)
Signed-off-by: Netanel Haber <nhaber@nvidia.com>
|
2026-04-24 11:53:23 +00:00 |
|
bnellnm
|
4a6dd1c3cc
|
[Bugfix] Fix DeepSeek V2-Lite Accuracy drop (#40673)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-04-23 18:11:37 -04:00 |
|
![gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
8f87eb4622
|
[Refactor] Clean up log once scope="local" (#40540)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-04-22 16:42:43 -04:00 |
|
bnellnm
|
809d83c2dc
|
[MoE Refactor] Combine MoERunnerBase + DefaultMoERunner (#40560)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-04-22 14:43:17 +00:00 |
|
Martin Hickey
|
3951d3eacd
|
[MyPy] Enable mypy for vllm/model_executor/layers/ (#40159)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-04-21 20:15:02 -07:00 |
|
 bnellnmandRobert Shaw
|
6867bcd076
|
[Bugfix] Replace code that disabled shared expert overlap (#39222)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-04-20 19:36:16 -04:00 |
|
 bnellnmandRobert Shaw
|
726efe177b
|
[MoE Refactor] Move the shared/fused expert output sum into MoERunnerBase (#35949)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-04-20 12:28:46 -04:00 |
|
bnellnm
|
19ec9a0a62
|
[MoE Refactor] Refactor ZeroExpertFusedMoE into new framework (#35549)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-04-14 16:11:20 -04:00 |
|
 bnellnmandRobert Shaw
|
e1e318af01
|
[MoE Refactor] Remove MoE DP chunking (#39107)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-04-14 09:48:05 -04:00 |
|
Richard Zou
|
f44afef6d6
|
[compile] Allow strings in custom ops without regressing compilation times (#38123)
Signed-off-by: Richard Zou <zou3519@gmail.com>
|
2026-04-10 07:26:37 +00:00 |
|
Richard Zou
|
6c749399b7
|
[BugFix] fix tests/kernels/moe/test_moe_layer.py (#39404)
Signed-off-by: Richard Zou <zou3519@gmail.com>
|
2026-04-09 08:48:59 -04:00 |
|
Andrey Talman
|
2111997f96
|
[release 2.11] Update to torch 2.11 (#34644)
|
2026-04-07 18:55:48 -07:00 |
|
 bnellnmandRobert Shaw
|
93bada494f
|
[MoE Refactor] Split of DefaultMoERunner class (#35326)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-04-06 12:41:59 -04:00 |
|
 
|
228023b3a5
|
[Bugfix][MoE] Fix 6-8% decode regression: prefer multi-stream shared expert overlap (#38990)
Signed-off-by: Martin Vit <martin@voipmonitor.org>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-04-05 10:28:31 -04:00 |
|
bnellnm
|
7cf56a59a2
|
[MoE Refactor] Make SharedExperts class for use with DefaultMoERunner (#35153)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-04-01 09:44:08 -04:00 |
|
bnellnm
|
9279c59a0e
|
[MoE Refactor] DefaultMoERunner simplifcation (#33049)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-03-19 15:07:44 -04:00 |
|
elvischenv
|
296839a1b0
|
[Perf] Eliminate padding and slicing op for GPT-OSS with Flashinfer MXFP4 MXFP8 MoE (#30647)
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
|
2026-03-18 15:01:26 +00:00 |
|
  
|
2754231ba3
|
[Kernel] Add FlashInfer MoE A2A Kernel (#36022)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Leo Tian <lctian@nvidia.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: root <root@lyris0267.lyris.clusters.nvidia.com>
|
2026-03-15 23:45:32 -07:00 |
|
 
|
d5af196c18
|
[2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
|
2026-03-13 09:25:33 -04:00 |
|
Kunshang Ji
|
53ec16a705
|
[Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145)
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-03-12 07:57:47 -07:00 |
|
Hongbin Guo
|
4bf533623b
|
[Doc] Fix duplicate words in comments (#36713)
Signed-off-by: Hongbin10 <jdmjdm1998@163.com>
|
2026-03-10 21:28:31 -07:00 |
|
 
|
97995f6376
|
[MoE Refactor] Create MK for TRTLLM Kernels (#32564)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
|
2026-03-03 10:39:50 -08:00 |
|