iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,099 Commits 3 Branches 0 Tags
429a3da6654facfa792995bbbc75ee64cf83be56
Commit Graph
436 Commits
Author SHA1 Message Date
Woosuk KwonandClaude Opus 5.5 e57bb4ace7 [Core] Remove the legacy Ray compiled-graph executor (#61712) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 18:13:27 -07:00
stefankoncarevic 24aa1191f4 [ROCm][Bugfix] Run the AITER AR+RMSNorm fallback test on the platform… (#61350) 
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
 2026-10-13 08:35:57 +00:00
Jiangyun ZhuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2cc2e68d1c [Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-13 00:58:59 -07:00
Mikko Tukiainen abad32a718 [ROCm][Perf] Fix Gemma AR+RMS prefill fallback (#58547) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Signed-off-by: Mikko Tukiainen <mikko.tukiainen@amd.com>
 2026-10-12 19:53:08 +00:00
Ilya Markov a281ad32f0 [EPLB] Add EPLB support to dspark draft models (DeepSeekV4) (#57549) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
 2026-10-12 10:03:09 -04:00
HubertCodexAo Shen
 		74f4404f58 [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots (#61000) 
Signed-off-by: Hubert Marek <163992334+hubert-marek@users.noreply.github.com>
Signed-off-by: Ao Shen <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Ao Shen <aoshen@inferact.ai>
 2026-10-12 01:44:12 -04:00
aoshen02 bd5bbce6e4 [RL] Release weight-transfer communicators without stopping inference (#61060) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
 2026-10-11 21:15:14 +08:00
Subhash Polisettiaoshen02Codex
 		3c062d391d [Bugfix] Support 0-dim tensors in packed weight transfer (#52235) 
Signed-off-by: subhash polisetti <subhashr161347@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-11 01:41:06 +00:00
aoshen02andClaude Opus 5.5 1dfe9fbdae [Core] Release FlashInfer allreduce workspace with --enable-nccl-comm-suspend (#59360) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 02:26:40 +00:00
Bolin SunClaudeNick Hill
 		04b02fc8a5 [Core] shm tensor arena for efficient cpu->gpu worker broadcast (#51207) 
Signed-off-by: Bolin Sun <bolins@nvidia.com>
Signed-off-by: <bolins@nvidia.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-10-09 16:13:24 -07:00
Juntian LiuClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		efaca9547d [Perf][DSv4.1] Exchange packed FP8 Engram rows across DP and feed wkv MXFP8 (#59923) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-09 15:30:18 +08:00
Mia OjedaandLucas Wilkinson 4611c2c3d2 [BugFix][DiffusionGemma][CPU] Copy async output snapshots instead of aliasing reused buffers (#59829) 
Signed-off-by: ojeda-e <eojeda@league.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-10-08 19:16:51 -07:00
Aarushi JainandAndreas Karatzas 9bbc581560 [Bugfix] Close MessageQueue zmq resources via finalizer to avoid exit (#60472) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-08 13:53:00 -05:00
Yifan QiaoCodexClaude Opus 5.5
 		ef63c23d35 [Bugfix][DSv4.1] Skip SWA bounded replay for KV loads that carry the window (#59197) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-07 15:28:42 -07:00
Samuel NordmannandMisha Goin 4a99ca40d6 [Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter (#49194) 
Signed-off-by: samnordmann <snordmann@nvidia.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-06 07:37:01 -07:00
LostFox11LostFox11CodexYangShuai52mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		15c1cc12ac [Bugfix][EPLB] Use group-local ranks for torch P2P transfers (#55804) 
Signed-off-by: LostFox11 <wangziyue17@huawei.com>
Co-authored-by: LostFox11 <wangziyue17@huawei.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: YangShuai52 <178856220+YangShuai52@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-06 07:44:57 +00:00
Elvir Crnčević ce49174247 [Bugfix] Fix FlashInfer all_reduce backend selection (#56891) 2026-10-03 22:18:58 -04:00
aoshen02andClaude Opus 5.5 5e56e9af2a [Feature] Release the CUDA graph pool on sleep (#59160) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 01:13:10 +00:00
DOCCA0andRobert Shaw 7dfe3338d5 [EPLB] Add contention-aware expert migration batching (#52641) 
Signed-off-by: DOCCA0 <2947323341@qq.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-03 00:13:50 +00:00
Ayaan Nadamal 84593955fa [Perf][MoE] Support fp8 combine in FlashInfer one-sided MoE all2all (#57995) 
Signed-off-by: Ayaan Nadamal <anadamal@nvidia.com>
 2026-10-02 16:29:16 -04:00
Julien Debache 111f71a6db [feat] FlashInfer CuteDSL MegaMoE integration (#54049) 
Signed-off-by: jdebache <jdebache@nvidia.com>
 2026-10-02 15:43:33 -04:00
djramic 167a81fa9c [CI] Fix ModelExpress handling in weight transfer tests (#59772) 
Signed-off-by: Djordje Ramic <djoramic@amd.com>
 2026-10-02 11:30:01 -05:00
tenotekandClaude Fable 5.1 443dccd3c9 [Bugfix][Multimodal] Fix reference counting of the SHM processor cache (#59695) 
Signed-off-by: tenotek <50898317+hdq66666@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 22:14:13 +08:00
Guanxin Liaoshen524Claude Opus 5.5aoshen02
 		a810316c08 [Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group (#59500) 
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 11:19:28 +08:00
Hyunjae Wooandaoshen02 5a661487e2 [Feature] Add native ModelExpress weight transfer backend (#58399) 
Signed-off-by: Hyunjae Woo <hwoo@nvidia.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-02 02:29:35 +00:00
Flora Feng 785b990116 [Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623) 
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-02 02:25:48 +00:00
Juntian LiuandClaude Opus 5.5 4c2d277643 [Perf][DSv4.1] Faster Engram host lookups: sorted rows, inline big lookups (#59327) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 11:54:44 +08:00
Clinton ThomasLucas BourtouleCyrus Leung
 		3d5625cc3c [Security] Authenticate shared-memory multimodal cache handles (#59357) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-30 05:16:26 -07:00
Andreas Karatzas 6ebb5bd64f [ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX (#59137) 
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
 2026-09-29 03:40:07 -05:00
Yifan QiaoandClaude Fable 5.1 4b2e1cfa7b [Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 (#58132) 
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-29 08:25:46 +00:00
Kevin H. LuuClaude Opus 5.5Lucas WilkinsonNick Hill
 		764413559a [Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace creation in eager mode (#58498) 
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-28 12:14:21 -07:00
Yongye ZhuandClaude Opus 5.5 77871126f9 [Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary (#58586) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-27 03:08:36 +08:00
qizixiOpenAI CodexClaude Opus 5.5Nick Hill
 		a4eb3f25d6 [Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-09-26 17:01:17 +08:00
Ke Wenmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>aoshen02
 		b6d2b4fc25 [RL] Add sharding-aware NCCL M2N weight transfer (#51520) 
Signed-off-by: Ke Wen <kwen@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-26 01:38:53 +00:00
Yongye ZhuandSummer Yang 442bd00d9f [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
 2026-09-25 03:27:21 +00:00
Hongxin Xuaoshen02OpenAI CodexRoger WangNick HillCodex
 		0b7f11a1ee [AuxOutput] Add block-keyed storage for routed-expert outputs (#45635) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 10:25:05 +00:00
Juntian LiuClaude Fable 5.1zjy0516
 		76ffb0d374 [Model][Engram] Share host tables across co-located DP replicas by default (#57651) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-09-21 08:26:56 +00:00
Woosuk KwonandCodex 9a70c233cd [Perf][DSV4.1] Fuse TP all-reduce with mHC input preparation (#57643) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-20 20:28:06 -07:00
YiSheng5andKunshang Ji 1c3ef2ad4c [XPU][UT]Fix test_mnnvl_alltoall device and distributed backend (#57591) 
Signed-off-by: yisustc <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-20 10:53:00 +08:00
YiSheng5andKunshang Ji f4fc5aae7f [XPU][UT]Using the device_control_env_var to restrict visible device on different platform (#57581) 
Signed-off-by: yisheng <yi.sheng@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-20 10:52:50 +08:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
Mikko TukiainenCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1df336c3ba [Perf][ROCm] Enable AITER QuickReduce + RMSNorm fusion (#48249) 
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 15:05:48 +00:00
Qiming Zhang e6b1a5e5e3 [XPU] Enable XPU eplb (#44987) 
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Signed-off-by: Qiming Zhang <qiming1.zhang@intel.com>
 2026-09-17 13:19:13 +08:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
AlexHuangOpenAI CodexItay Etelis
 		0f8fa53acf [Bugfix][KV Offload] Check cgroup memory before SHM allocation (#54014) 
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: AlexHuang <alex.tech.lab@outlook.com>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
 2026-09-16 09:17:50 +03:00
Andreas KaratzasandCodex a7576447b8 [ROCm][Bugfix] Initialize Ray NIXL agents for sharded RDT (#56687) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-15 00:34:21 +00:00
Yongye Zhu 5372e72a98 [Perf][DSv4.1] Fold the mHC post block into the delayed pre projection (#56633) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-09-14 18:57:37 +00:00
Bugen Zhao cf1584f373 [Refactor] Normalize DeepSeek V4.1 model package naming (#56741) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-13 23:17:50 -07:00
Juntian LiuRoger Wangywang96 <>Woosuk KwonCodex
 		d2d649e674 [DS V4.1][Engram] Support async prefetch for offloaded engram lookups and engram DP sharding (#56512) 
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: ywang96 <>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-13 02:44:23 -07:00