iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
106 Commits
Author SHA1 Message Date
vllm-agentandKevin Luu d1e5e66ee3 [Bugfix] Restore portable all2all backend default (#53952) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
 2026-08-26 16:19:49 -07:00
Artem Perevedentsev 2cd6c664c1 [MoE] enable all2all fi_one_sided by default (#53311) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-08-26 17:03:45 +05:00
Giancarlo Delfin d154d90d6c [Model Runner V2] batch-sharded sample (#50465) 
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
 2026-08-24 15:49:57 -07:00
Lucas WilkinsonOpenAI CodexClaude Opus 5
 		72aedcc426 [DCP] Default query replication for GLM sparse attention (#50382) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 09:07:05 -07:00
Itay Alroy a2257f95b7 [Elastic EP] Reduce eager-mode reconfiguration downtime (#51885) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-08-19 02:13:06 +00:00
Bo Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> c296851a7d [MoE] Refine FlashInfer one-sided All2All integration (#51924) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 00:18:14 +00:00
aoshen02andOpenAI Codex 22a175921c [BugFix] Reject invalid data-parallel RPC ports (#51469) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-08 12:23:26 +08:00
aoshen02andAndy Lo 8cfa01cdd2 [BugFix] Dense multinode DP rescope with regression test (#49212) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Andy Lo <andy@mistral.ai>
 2026-08-05 18:12:03 -07:00
+13
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodexWei Zhao
 		aeeb36b1f1 [New model] Kimi K3 (#50000) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
 2026-07-30 03:49:58 -07:00
Itay Alroy 35efdf6b34 [Elastic EP] Async preparation (#47288) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-07-28 05:19:13 -07:00
fangyuchuzWaNg3a798347923TianZhuo205150940a798347923w00689259zWaNg3yzchang-plusJade Zheng
 		0b0bd2b5f6 [Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments (#44428) 
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Signed-off-by: a798347923 <2645302020@qq.com>
Signed-off-by: TianZhuo <2770730562@qq.com>
Signed-off-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Signed-off-by: 205150940 <112750056+205150940@users.noreply.github.com>
Signed-off-by: w00689259 <wangzhuo66@huawei.com>
Signed-off-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Signed-off-by: zWaNg3 <389750525@qq.com>
Signed-off-by: yzchang-plus <1078477584@qq.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Co-authored-by: a798347923 <2645302020@qq.com>
Co-authored-by: TianZhuo <2770730562@qq.com>
Co-authored-by: 205150940 <112750056+205150940@users.noreply.github.com>
Co-authored-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Co-authored-by: w00689259 <wangzhuo66@huawei.com>
Co-authored-by: zWaNg3 <389750525@qq.com>
Co-authored-by: yzchang-plus <1078477584@qq.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
 2026-07-25 11:49:09 -04:00
Andreas Karatzas 16aca639b7 [ROCm] Upgrade NIXL and UCX (#49251) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-07-21 22:30:02 -05:00
Lucas WilkinsonCodexOpenAI CodexMatthew Bonanni
 		b6ff8a2f50 [Core] Add MRV2 virtual-batch PCP for MLA (#46570) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-07-19 02:53:15 +00:00
passtoor-agi c4dd6d78fd Fix: Restore data_parallel_size > 1 for use_sequence_parallel_moe (#48849) 
Signed-off-by: passtoor-agi <305788622+passtoor-agi@users.noreply.github.com>
 2026-07-17 10:27:50 -04:00
Wentao Ye 1ff9429655 [CI Bug] Fully solve accuracy issue for DSv3.2 + MTP + Sequence Parallel (#48036) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-14 10:00:24 -04:00
Wentao Ye 3f99883d97 [CI Bug Fix] Temp fix for v3.2 accuracy (#47902) 2026-07-07 16:36:03 -04:00
Wentao YeandCanlin Guo b6cc46ec3b [Feature] Support sequence parallel without the need for DP, 1.9%~5.0% E2E Throughput Improvement (#47070) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-07-05 05:41:30 -07:00
Ilya MarkovandMarkov Ilya ac614587f5 [EPLB] Enable nixl eplb communicator for elastic ep (#45013) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
 2026-06-22 10:54:08 -07:00
Tyler Michael SmithClaudeCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>kourosh hakhamaneshi
 		ebfbcfe46a Stop setting CUDA_VISIBLE_DEVICES internally in vLLM, add device_ids arg (#45026) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: kourosh hakhamaneshi <kouroshHakha@users.noreply.github.com>
 2026-06-20 13:38:10 -07:00
Ilya Markov 6471ec75bd [EPLB] Reject NCCL-based EPLB communicators with async EPLB (#44978) 
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
 2026-06-10 19:51:27 +00:00
Tyler Michael SmithandClaude Opus 4.6 e2f993dc41 [WideEP] Integrate DeepEP v2 (#41183) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-06-08 18:07:29 -07:00
JianweiZhengandWentao Ye 99ef652907 [Bugfix] Reject non-positive values for ParallelConfig int knobs (#44057) 
Signed-off-by: jwzheng96 <jianweizheng@pku.edu.cn>
Signed-off-by: JianweiZheng <32029023+jwzheng96@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-06-04 11:46:50 -04:00
Ilya MarkovMarkov IlyaTyler Michael Smith
 		4aaba00f92 [EPLB] Make async EPLB default (#43219) 
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2026-05-29 18:07:16 +00:00
Vadim GimpelsonandCursor bf18d7e0b4 [Misc][NUMA] Auto-bind to PCT priority cores on DGX B300 + widen EngineCore across shard NUMA nodes (#43270) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Cursor <noreply@cursor.com>
 2026-05-29 10:07:44 +08:00
jatseng-ai 05c50c721e [ROCm] mori: add InterNodeV1LL inter-node kernel selection via VLLM_MORI_INTERNODE_KERNEL (#41751) 
Signed-off-by: jatseng-ai <jatseng@amd.com>
 2026-05-28 00:33:32 +08:00
Ilya MarkovandMarkov Ilya d3a563501b [EPLB] Change default EPLB communicator (#43110) 
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
 2026-05-22 09:43:27 -04:00
fangyuchuzWaNg3mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e26e1f0928 [Feature] Add --cpu-distributed-timeout-seconds CLI Option for CPU Process Group Timeout (#42968) 
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Signed-off-by: zWaNg3 <389750525@qq.com>
Co-authored-by: zWaNg3 <389750525@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-21 15:42:07 -07:00
Wentao Ye 577b9623e6 [Bug] Fix status update address for non-MOE model within external dp mode (#40839) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-04 16:37:16 -07:00
Aaron HaoJunjie ZhangNick Hill
 		0335316a9b [BUG] Two phase pause to prevent deadlock (#39366) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Signed-off-by: Aaron Hao <ahao@anyscale.com>
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-04-29 17:51:03 -04:00
Roy Wang 9e92de51c6 [Bugfix] Exclude numa_bind fields from ParallelConfig DP hash (#41098) 
Signed-off-by: yasong <yasong.wang@inferact.ai>
 2026-04-28 15:52:54 +08:00
Ilya Markov 50dd4cb427 [EPLB] Add nixl-based eplb communicator (#36276) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
 2026-04-20 10:24:23 +00:00
bnellnmandRobert Shaw e1e318af01 [MoE Refactor] Remove MoE DP chunking (#39107) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-04-14 09:48:05 -04:00
Shengqi ChenJason LiRoger Wang
 		75e01a39a1 [Feature] NUMA binding support for GPU workers (#38635) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
Co-authored-by: Jason Li <jasonlizhengjian@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-04-08 09:55:24 -07:00
wliao2Harry MellorKunshang Ji
 		4dfad17ed1 replace cuda_device_count_stateless() to current_platform.device_count() (#37841) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-31 22:32:54 +08:00
Ilya MarkovandMarkov Ilya abdbb68386 [EPLB] Add alternative communication for EPLB weight exchange (#33176) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
 2026-03-31 08:17:12 -04:00
Mateusz Sokół 8278825b57 DOC: TPU mention fix (#38129) 
Signed-off-by: Mateusz Sokół <mat646@gmail.com>
 2026-03-31 03:27:56 -07:00
Ilya Markov 12701e8af2 [EPLB] Optmize eplb mapping and record in router for prefill (#36261) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-03-30 19:48:33 +00:00
Harry Mellorandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 21d2b53f88 Remove need for explicit \n in docstring lists for --help formatting (#38350) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-27 08:38:00 -07:00
Harry Mellor d215d1efca [Mypy] Better fixes for the mypy issues in vllm/config (#37902) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-25 06:14:43 -07:00
RanranandWentao Ye dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) 
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-23 18:31:14 -04:00
bnellnm 91be5f9be3 [MoE Refactor] Rename "naive" all2all backend (#36294) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-03-19 15:50:34 -04:00
Itay Alroy de1a86b7de elastic_ep: Fix stateless group port races (#36330) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
 2026-03-18 14:36:18 +00:00
Roy WangandClaude Opus 4.6 761e0aa7a0 [Performance] Add --enable-ep-weight-filter CLI option (#37351) 
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-18 09:36:55 +08:00
leo-cf-tianwzhao18Stefano Castagnettaroot
 		2754231ba3 [Kernel] Add FlashInfer MoE A2A Kernel (#36022) 
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Leo Tian <lctian@nvidia.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: root <root@lyris0267.lyris.clusters.nvidia.com>
 2026-03-15 23:45:32 -07:00
Itay AlroyYongji WuRon Tourgeman
 		d5af196c18 [2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-03-13 09:25:33 -04:00
Shiyan DengandLu Fang 03a49bb8f0 [Feature] Add --distributed-timeout-seconds CLI option (#36047) 
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
 2026-03-05 20:57:51 -08:00
Harry Mellor 17dc9c7fc9 [CI] Bump mypy version (#34950) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 20:55:11 +00:00
sungsoo haClaude Opus 4.6Lucas WilkinsonHarry Mellor
 		6cb901093f [Core] Add All-to-All communication backend for DCP (#34883) 
Signed-off-by: Sungsoo Ha <sungsooh@nvidia.com>
Signed-off-by: sungsoo ha <hasungsoo@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 10:01:57 -05:00
Ilya Markov b2d8b422b2 [EPLB] Enforce sync eplb for NCCL-based all2all backend (#35212) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
 2026-02-28 05:47:12 +00:00
Itay AlroyYongji WuTyler Michael SmithRon Tourgeman
 		dea268336f [1/N] Elastic EP Milestone 2 (#34861) 
Signed-off-by: Yongji Wu <wuyongji317@gmail.com>
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Ron Tourgeman <rtourgeman@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-02-28 04:46:42 +00:00