iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,991 Commits 3 Branches 0 Tags
c19c263ac8ea3fd6e1d2d689f0e4e725855d89cf
Commit Graph
334 Commits
Author SHA1 Message Date
Misha GoinandClaude d79b52b182 [Misc] Consolidate csrc shared headers and prune dead code (#60721) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-12 09:46:22 -07:00
MattClaude Opus 5.5Andreas Karatzas
 		e0317b28ce [Feature] Support batch invariance on ROCm (#52231) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-12 10:53:26 -05:00
AMD-yanfeiwangandaoshen02 e3bbaf2369 [ROCm][DSv4] Fuse the shared expert into AITER MegaMoE (#61004) 
Signed-off-by: Yanfei Wang <yanfei.wang@amd.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-12 15:53:16 +08:00
HubertCodexAo Shen
 		74f4404f58 [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots (#61000) 
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
 2026-10-12 01:44:12 -04:00
aoshen02andClaude Opus 5.5 1dfe9fbdae [Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend (#59360) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 02:26:40 +00:00
Bolin SunClaudeNick Hill
 		04b02fc8a5 [Core] shm tensor arena for efficient cpu->gpu worker broadcast (#51207) 
Signed-off-by: Bolin Sun <bolins@nvidia.com>
Signed-off-by: <bolins@nvidia.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-10-09 16:13:24 -07:00
Misha GoinandClaude 3766a67a32 [WideEP] Explain how to fix a too-old NCCL for DeepEP v2 (#60449) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-09 12:34:58 -07:00
aoshen02Claude Fable 5.1indianspeedster
 		a6f43e57d6 [ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
 2026-10-09 08:29:55 +00:00
Aarushi JainandAndreas Karatzas 9bbc581560 [Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit (#60472) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-08 13:53:00 -05:00
Micah WilliamsonandFangzhou Ai f6e3944e31 [ROCm][DSv4] AITER MegaMoEV2 Integration For DeepSeek V4 (#59685) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com>
 2026-10-07 09:35:53 -05:00
Samuel NordmannandMisha Goin 4a99ca40d6 [Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter (#49194) 
Signed-off-by: samnordmann <snordmann@nvidia.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 07:37:01 -07:00
Elvir Crnčević ce49174247 [Bugfix] Fix FlashInfer all_reduce backend selection (#56891) 2026-10-03 22:18:58 -04:00
aoshen02andClaude Opus 5.5 5e56e9af2a [Feature] Release the CUDA graph pool on sleep (#59160) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 01:13:10 +00:00
Ayaan Nadamal 84593955fa [Perf][MoE] Support fp8 combine in FlashInfer one-sided MoE all2all (#57995) 
Signed-off-by: Ayaan Nadamal <anadamal@nvidia.com>
 2026-10-02 16:29:16 -04:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
tenotekandClaude Fable 5.1 443dccd3c9 [Bugfix][Multimodal] Fix reference counting of the SHM processor cache (#59695) 
Signed-off-by: tenotek <50898317+hdq66666@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 22:14:13 +08:00
Flora Feng 785b990116 [Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623) 
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-02 02:25:48 +00:00
Clinton ThomasLucas BourtouleCyrus Leung
 		3d5625cc3c [Security] Authenticate shared-memory multimodal cache handles (#59357) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-30 05:16:26 -07:00
Tyler Michael SmithandCodex 50239dfc4d [WideEP] Change DeepEPv2 to auto select hybrid mode by default (#57991) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 18:36:30 -04:00
Kevin H. LuuClaude Opus 5.5Lucas WilkinsonNick Hill
 		764413559a [Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode (#58498) 
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-28 12:14:21 -07:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Yongye ZhuandSummer Yang 442bd00d9f [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-09-25 03:27:21 +00:00
Lucas WilkinsonQiuChunshuoClaude Fable 5.1CodexRobert ShawNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e6dc16cebd [PCP] Support prefill context parallelism with data parallelism (#57075) 
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-23 22:32:36 -04:00
Tony LinandFlora Feng e4340e41c9 [Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models (#55881) 
Signed-off-by: Tony Lin <tony.lin@intel.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-22 11:06:54 -04:00
YiSheng5andKunshang Ji 17e50b9b76 [XPU][UT]Bugfix when the process can't see all the world_size meet accuracy issue. (#57779) 
Signed-off-by: yisheng <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-21 06:50:25 +08:00
Kevin Liandkevinli 911548247a [Distributed][MoonEP] BF16 integration of MoonEP balanced EP backend (#52101) 
Signed-off-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
Co-authored-by: kevinli <306243285+kaijunli-infr@users.noreply.github.com>
 2026-09-18 10:34:01 -04:00
drakoshaClaudeColton OttleyOr Ozeri
 		40b40d1d39 [Bugfix][KV Offload] Register the offload region in chunks (#51081) 
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Colton Ottley <coltonottley@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-09-17 13:19:56 +03:00
Robert ShawandRobert Shaw c38f92aae7 [A2A] Allow EPLB + SharedExpert Overlap + DeepEPv2 (#57236) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-17 04:02:39 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
AlexHuangOpenAI CodexItay Etelis
 		0f8fa53acf [Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-09-16 09:17:50 +03:00
MattandClaude Opus 4.8 72d4d83157 [ROCm][Bugfix] Fix elastic EP scaling deadlock (#56610) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-09-12 23:09:19 +00:00
Itay Alroy 658c8131c7 [Elastic EP] Reuse CUDA graphs across reconfiguration (#54985) 2026-09-12 10:26:39 -04:00
Nicolò LucchesiandHarry Mellor 8c1d1c2974 [Misc] Log FlashInfer allreduce workspace init failure as error (#55127) 
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-11 09:35:56 +00:00
Simon Danielsson ae48466cf3 [Perf][ROCm] Add AITER custom AG/RS (DP only) (#48247) 
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
 2026-09-11 13:19:57 +08:00
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 3116c5d06b [Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-09 07:32:34 -07:00
Taoyu ZhuandShanshan Shen ce6c241ddc [ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path (#54787) 
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
 2026-09-08 07:21:49 +00:00
dharak-cohereandCursor d0e695a91b [Distributed] Support pre-shared ncclUniqueId rendezvous for weight transfer (#53784) 
Signed-off-by: Dharak Kharod <dharak.kharod@cohere.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-01 14:55:57 +08:00
aoshen02aoshen02Claude Fable 5
 		4ac452ad98 [Core] Release NCCL communicator memory in sleep mode (#51485) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-01 14:31:16 +08:00
Gabriel WuYongye Zhumergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		446c769482 [Distributed] Add opt-in FlashInfer PCIe IPC all-reduce backend (#53576) 
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-01 03:34:56 +00:00
+5
Harry Huangzjy0516KungYorkThien TranYifan Qiaoandyluo7Isotr0pyandyluo7OpenAI Codexlishunyang12Vadim GimpelsonIsotr0pyLucas WilkinsonClaude Opus 5OpenAI Codex
 		e126687a9a [Model] Support Qwen3.8-Flash-Next (#53896) 
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-31 05:57:56 +00:00
JulianCodexSummer Yang
 		99013d77d3 [Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability (#53253) 
Signed-off-by: JulianZJN <148066842+JulianZJN@users.noreply.github.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-29 08:49:24 +08:00
Xuan-1998andVladimir Aerov 026d5af7f4 [Bugfix] Fix ncclCommQueryProperties heap overflow with NCCL >= 2.31 (#53008) 
Signed-off-by: Xuan Jiang <xuanj@amazon.com>
Co-authored-by: Vladimir Aerov <vaerov@amazon.com>
 2026-08-29 00:25:00 +00:00
Khushali Desai c7b0467c25 buffer size insuffient Dspark sd for FlashInfer MNNVL allreduce (#50932) 2026-08-27 11:01:08 -04:00
Artem Perevedentsev 2cd6c664c1 [MoE] enable all2all fi_one_sided by default (#53311) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-08-26 17:03:45 +05:00
Jee Jee Li 88e1b11313 [Perf] Tune FlashInfer all-reduce thresholds for single-node TP8 on SM103 (#53606) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-08-26 01:46:56 +00:00
Summer Yang e6e1af4ca1 [Perf] Tune FlashInfer all-reduce selection on SM103 (#53318) 
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
 2026-08-24 05:19:59 +00:00
shijuzhaoandshijuzhao cda3868f5d [Bugfix] Deterministic MoE combine (reduce_scatterv) under VLLM_BATCH_INVARIANT (#45683) 
Signed-off-by: shijuzhao <shijuzhao@tencent.com>
Co-authored-by: shijuzhao <shijuzhao@tencent.com>
 2026-08-21 21:41:31 +08:00
Woosuk KwonandOpenAI Codex 6a962071bd [Distributed] Enable FlashInfer all-reduce by default (#52998) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 18:17:43 -07:00
Bo Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c296851a7d [MoE] Refine FlashInfer one-sided All2All integration (#51924) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 00:18:14 +00:00