Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
Yongye Zhu and Summer Yang
442bd00d9f
[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 ( #55072 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-09-25 03:27:21 +00:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Tony Lin and Flora Feng
e4340e41c9
[Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models ( #55881 )
...
Signed-off-by: Tony Lin <tony.lin@intel.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-22 11:06:54 -04:00
YiSheng5 and Kunshang Ji
17e50b9b76
[XPU][UT]Bugfix when the process can't see all the world_size meet accuracy issue. ( #57779 )
...
Signed-off-by: yisheng <yi.sheng@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-21 06:50:25 +08:00
Kevin Li and kevinli
911548247a
[Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend ( #52101 )
...
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com >
2026-09-18 10:34:01 -04:00
40b40d1d39
[Bugfix][KV Offload] Register the offload region in chunks ( #51081 )
...
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Colton Ottley <coltonottley@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-09-17 13:19:56 +03:00
Robert Shaw and Robert Shaw
c38f92aae7
[A2A] Allow EPLB + SharedExpert Overlap + DeepEPv2 ( #57236 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-17 04:02:39 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
0f8fa53acf
[Bugfix][KV Offload] Check cgroup memory before SHM allocation ( #54014 )
...
Signed-off-by: Alex <jihui.huang@daocloud.io >
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com >
Signed-off-by: Alex <alex.tech.lab@outlook.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
2026-09-16 09:17:50 +03:00
Matt and Claude Opus 4.8
72d4d83157
[ROCm][Bugfix] Fix elastic EP scaling deadlock ( #56610 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-09-12 23:09:19 +00:00
Itay Alroy
658c8131c7
[Elastic EP] Reuse CUDA graphs across reconfiguration ( #54985 )
2026-09-12 10:26:39 -04:00
Nicolò Lucchesi and Harry Mellor
8c1d1c2974
[Misc] Log FlashInfer allreduce workspace init failure as error ( #55127 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-11 09:35:56 +00:00
Simon Danielsson
ae48466cf3
[Perf][ROCm] Add AITER custom AG/RS (DP only) ( #48247 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-11 13:19:57 +08:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3116c5d06b
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism ( #54371 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
Taoyu Zhu and Shanshan Shen
ce6c241ddc
[ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path ( #54787 )
...
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-08 07:21:49 +00:00
dharak-cohere and Cursor
d0e695a91b
[Distributed] Support pre-shared ncclUniqueId rendezvous for weight transfer ( #53784 )
...
Signed-off-by: Dharak Kharod <dharak.kharod@cohere.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-01 14:55:57 +08:00
4ac452ad98
[Core] Release NCCL communicator memory in sleep mode ( #51485 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-01 14:31:16 +08:00
446c769482
[Distributed] Add opt-in FlashInfer PCIe IPC all-reduce backend ( #53576 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 03:34:56 +00:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
99013d77d3
[Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability ( #53253 )
...
Signed-off-by: JulianZJN <148066842+JulianZJN@users.noreply.github.com >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
2026-08-29 08:49:24 +08:00
Xuan-1998 and Vladimir Aerov
026d5af7f4
[Bugfix] Fix ncclCommQueryProperties heap overflow with NCCL >= 2.31 ( #53008 )
...
Signed-off-by: Xuan Jiang <xuanj@amazon.com >
Co-authored-by: Vladimir Aerov <vaerov@amazon.com >
2026-08-29 00:25:00 +00:00
Khushali Desai
c7b0467c25
buffer size insuffient Dspark sd for FlashInfer MNNVL allreduce ( #50932 )
2026-08-27 11:01:08 -04:00
Artem Perevedentsev
2cd6c664c1
[MoE] enable all2all fi_one_sided by default ( #53311 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-08-26 17:03:45 +05:00
Jee Jee Li
88e1b11313
[Perf] Tune FlashInfer all-reduce thresholds for single-node TP8 on SM103 ( #53606 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-26 01:46:56 +00:00
Summer Yang
e6e1af4ca1
[Perf] Tune FlashInfer all-reduce selection on SM103 ( #53318 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-24 05:19:59 +00:00
shijuzhao and shijuzhao
cda3868f5d
[Bugfix] Deterministic MoE combine (reduce_scatterv) under VLLM_BATCH_INVARIANT ( #45683 )
...
Signed-off-by: shijuzhao <shijuzhao@tencent.com >
Co-authored-by: shijuzhao <shijuzhao@tencent.com >
2026-08-21 21:41:31 +08:00
Woosuk Kwon and OpenAI Codex
6a962071bd
[Distributed] Enable FlashInfer all-reduce by default ( #52998 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-19 18:17:43 -07:00
Bo Li and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c296851a7d
[MoE] Refine FlashInfer one-sided All2All integration ( #51924 )
...
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-18 00:18:14 +00:00
f7ef489e93
[Bugfix] Fix ZMQ port TOCTOU race in shm_broadcast MessageQueue ( #50960 )
...
Signed-off-by: RTCartist <wangshengb@buaa.edu.cn >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: RTCartist <wangshengb@buaa.edu.cn >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-07 18:19:02 -07:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Michael Goin and OpenAI Codex
90329913e9
[UX] Reduce startup log noise ( #50590 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-31 13:04:34 -07:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
bugkeep and Claude
48aa8d8d75
[Bugfix] Prevent stale multiproc RPC deadlines from becoming unbounded waits ( #41357 )
...
Signed-off-by: bugkeep <1921817430@qq.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-29 16:41:34 -07:00
5fa0154448
[Rubin] Enable NVLink all-reduce paths on SM107 ( #49647 )
...
Signed-off-by: Zachary Aristei <zaristei@nvidia.com >
Co-authored-by: Sahithi Chigurupati <chigurupati.sahithi@gmail.com >
Co-authored-by: hlu1 <14827759+hlu1@users.noreply.github.com >
2026-07-29 20:53:25 +00:00
Ruinan Ma and Nick Hill
a07fac758f
[Perf] Zero-copy torch.Tensor pickling in shm_broadcast MessageQueue ( #48442 )
...
Signed-off-by: Ruinan Ma <r7ma3088@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-28 16:28:28 -07:00
Itay Alroy
35efdf6b34
[Elastic EP] Async preparation ( #47288 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
2026-07-28 05:19:13 -07:00
831d3848f1
[Core] Fail fast when /dev/shm is too small for the shm ring buffer ( #48879 )
...
Signed-off-by: Dr Andrea Tassi <andrea@verticular.uk >
Co-authored-by: Dr Andrea Tassi <andrea@verticular.uk >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-27 19:48:35 +00:00
rongfu.leng and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
81962bb699
[Bugfix]Reject invalid FlashInfer MNNVL workspaces ( #49043 )
...
Signed-off-by: lengrongfu <lenronfu@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 08:12:23 -04:00
Schwinn Saereesitthipitak
b5b61c622c
[Core][Distributed] Add process-checkpoint lifecycle hooks for communicators (starting with Flashinfer) ( #46877 )
...
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com >
2026-07-26 14:47:50 -04:00
0b0bd2b5f6
[Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments ( #44428 )
...
Signed-off-by: fangyuchu <fangyuchu@qq.com >
Signed-off-by: a798347923 <2645302020@qq.com >
Signed-off-by: TianZhuo <2770730562@qq.com >
Signed-off-by: a798347923 <39047817+a798347923@users.noreply.github.com >
Signed-off-by: 205150940 <112750056+205150940@users.noreply.github.com >
Signed-off-by: w00689259 <wangzhuo66@huawei.com >
Signed-off-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com >
Signed-off-by: zWaNg3 <389750525@qq.com >
Signed-off-by: yzchang-plus <1078477584@qq.com >
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com >
Co-authored-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com >
Co-authored-by: a798347923 <2645302020@qq.com >
Co-authored-by: TianZhuo <2770730562@qq.com >
Co-authored-by: 205150940 <112750056+205150940@users.noreply.github.com >
Co-authored-by: a798347923 <39047817+a798347923@users.noreply.github.com >
Co-authored-by: w00689259 <wangzhuo66@huawei.com >
Co-authored-by: zWaNg3 <389750525@qq.com >
Co-authored-by: yzchang-plus <1078477584@qq.com >
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com >
2026-07-25 11:49:09 -04:00
5d8e90a966
[WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image ( #45321 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Signed-off-by: Tyler Michael Smith <tyler@vllm.ai >
Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Codex <noreply@openai.com >
2026-07-24 13:00:02 -07:00
10c75477b0
[Bugfix][Core] shm_broadcast: bound idle reader waits and release read slots ( #45224 )
...
Signed-off-by: Chaemin Lim <chaemin.lim@mangoboost.io >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Edwin Lim <edwin.lim@mangoboost.io >
Co-authored-by: Jaeyoun Kim <jaeyoun.kim@mangoboost.io >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-23 18:13:29 +08:00
b6ff8a2f50
[Core] Add MRV2 virtual-batch PCP for MLA ( #46570 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-07-19 02:53:15 +00:00
Samuel Nordmann and Codex
d08eebad16
[Perf][MoE] Write FlashInfer combine into final output ( #47156 )
...
Signed-off-by: snordmann <snordmann@nvidia.com >
Co-authored-by: Codex <codex@openai.com >
2026-07-16 17:21:03 +03:00
Mohammad Miadh Angkad
75fe92a316
[Distributed][Perf] Enable FlashInfer MNNVL allreduce RMS quant fusion ( #48064 )
...
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
2026-07-13 15:02:59 +08:00
Rehan Khan and Li, Jiang <jiang1.li@intel.com>
9e57de7197
[CPU] Create Proper Numa topology for s390x ( #40714 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-07-13 12:58:43 +08:00
Fangzhou Ai and Cursor
775c1589ea
[Bugfix][ROCm] Keep TP all_gather on base-class collective ( #48446 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-07-13 03:53:53 +00:00
Yan Xu and Jingyi Yang
95ed0feaa5
DCP supports hybrid attention ( #40996 )
...
Signed-off-by: YanXu <yancey.yx@alibaba-inc.com >
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com >
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com >
2026-07-09 21:34:45 -07:00