iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
19,362 Commits 3 Branches 0 Tags
f1899b2ffdf87598350c416a074c8c291c845902
Commit Graph
67 Commits
Author SHA1 Message Date
Max HuRoger WangShang Wang
 		5a65ba5f17 [Refactor] Move iteration logging to the frontend (#46647) 
Signed-off-by: maxyanghu <hyoung2991@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Shang Wang <shangw@nvidia.com>
 2026-07-15 17:59:05 -07:00
Harry Mellor e24d1b24fe Fix Transformers modeling backend usage stats (#47472) 2026-07-02 12:51:23 -07:00
Tahsin TunanBugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b8cb75b149 [Rust Frontend] Add static HTTPS and mTLS support for HTTP and gRPC (#45890) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-30 01:45:59 +00:00
Kaiand图灵 9b215ae60b [Rust Frontend] Forward VLLM_ENGINE_READY_TIMEOUT_S via --args-json (#44610) 
Signed-off-by: kai <kai@example.com>
Co-authored-by: 图灵 <tuling.wk@alibaba-inc.com>
 2026-06-25 07:25:08 +00:00
Nick Hill 7df3d7dada [Core] Ensure memory is pinned prior to async h2d copy (#45424) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-20 20:02:24 -07:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e2c58570ea [Rust Frontend] Support hybrid/external DP LB in Python supervised bootstrap (#45805) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-17 07:32:40 +00:00
Zachary XiandCursor 3f627ebef7 [Misc] usage_stats: report more engine, spec-decode, and EP config (#44595) 
Signed-off-by: Zach Xi <zachary.xi@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-08 15:20:00 -07:00
Wentao Ye 41a4829f22 [Logs Refactor] Optimize shutdown logs, easier to follow and consistent (#43707) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-06-04 14:36:32 -04:00
wang.yuqi d01d0b4646 [Frontend] Consolidate online serving utils. (#44479) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-04 06:49:31 +00:00
Wentao Ye be4062fd6c [Bug] Fix tests/distributed/test_elastic_ep.py - assert False (#43813) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-28 11:00:56 -04:00
Vadim GimpelsonandNick Hill d6b48f928f [BugFix] Fix hard-coded timeout for multi-API-server startup (#43768) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-05-28 00:09:13 -07:00
Vadim Gimpelsonandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 812e7e7364 [Bugfix][V1] Fix TOCTOU race causing intermittent EADDRINUSE on multi-API-server DP startup (#42585) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-26 14:06:00 -07:00
Andreas Karatzas 5940590855 [ROCm][CI] Stabilize 400 error return code for invalid schema inputs (#43016) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-24 10:06:49 +00:00
Nick HillandBugen Zhao f2ace1d57d [Frontend][RFC] Rust front-end integration (#40848) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-05-21 12:24:48 +08:00
fangyuchuandNick Hill b5e608258e [Refactor] Unify engine process monitoring in engine manager and add Ray backend support (#35862) 
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-30 10:16:09 -07:00
Nick Hill 72cad44d3c [Frontend] Move APIServerProcessManager target server fn (#38115) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-03-25 18:14:41 +00:00
Brandon PelfreyandNick Hill 80b70884eb Add tensor IPC transfer mechanism for multimodal data (#32104) 
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Signed-off-by: Brandon Pelfrey <brandonpelfrey@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-21 20:10:20 +00:00
Mark McLoughlinClaude Sonnet 4.5Nick Hill
 		7afe0faab1 [Frontend][Core] Re-add shutdown timeout - allowing in-flight requests to finish (#36666) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-03-13 12:10:06 -07:00
Mark McLoughlin 234860399b [Frontend][Core] Revert "Add shutdown timeout" (#34730 and #36270) (#36628) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2026-03-10 06:20:41 -07:00
Mark McLoughlinandClaude Sonnet 4.5 27066d1b2b [Frontend][Core] Add shutdown timeout - allowing in-flight requests to finish (#34730) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-03-05 22:04:31 -08:00
杨朱 · KikiandClaude Opus 4.5 1a7894dbdf [Misc] Replace Optional[X] with X | None syntax (#33332) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-30 01:56:59 -08:00
Max HuandMax Hu 6ebe34d6fa [Feature] Add iteration level logging and enhance nvtx marker (#31193) 
Signed-off-by: Max Hu <maxhu@nvidia.com>
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>
 2026-01-09 00:13:39 +00:00
Nick Hill 80b6080ddc [BugFix] Fix async scheduling + chunked prefill + preemption (#28787) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-11-17 06:46:46 +08:00
Cyrus Leung 6ebffafbb6 [Misc] Clean up more utils (#27567) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 15:30:38 +00:00
Andrew Sansom ff93cc8c84 [CORE] Support Prefix Caching with Prompt Embeds (#27219) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-10-22 22:18:07 -07:00
iAmir97andiAmir97 7a6c8c3fa1 [Chore] Separate out vllm.utils.network_utils (#27164) 
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
 2025-10-19 03:06:32 -07:00
Tyler Michael Smith 579d2e5458 [WideEP][P/D] Add usage stats for DP+EP and KV Connector (#26836) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-10-14 23:51:54 +00:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor 6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Naman Lalit 4e33a7ea85 [Bugfix] Optimize CpuGpuBuffer initialization (#25447) 
Signed-off-by: Naman Lalit <nl2688@nyu.edu>
 2025-09-27 08:07:36 +08:00
Corey Lowman c85d75cf08 Add VLLM_NVTX_SCOPES_FOR_PROFILING=1 to enable nvtx.annotate scopes (#25501) 
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
 2025-09-23 22:50:09 +00:00
Boyuan FengandHarry Mellor 94e6b2d55f Allow users to specify kv cache memory size (#21489) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-11 13:41:07 +00:00
Bangsheng TangandBangsheng Tang 848562bd49 break execute_model in gpu_model_runner into sub-functions for custom scopes (#24265) 
Co-authored-by: Bangsheng Tang <bangsheng@meta.com>
 2025-09-06 14:02:47 -07:00
Andrew Sansom 305a1cc0d2 refactor: Turn GPUModelRunner.inputs_embeds to a CpuGpuBuffer (#24345) 
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
 2025-09-05 23:01:23 -07:00
Woosuk Kwon 04ff1e43fb [Misc] Move CpuGpuBuffer to vllm/v1/utils.py (#23728) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-08-27 03:25:00 -07:00
ZiTian.Zhao 49bcd893e7 [refactor] improve ConstantList exception specificity (#22156) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
 2025-08-03 22:14:49 -07:00
Nick Hill 8d524ce79f [BugFix] Improve internal DP load balancing (#21617) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-01 19:45:27 -07:00
Nick Hill 7234fe2685 [Misc] Rework process titles (#21780) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-29 05:14:47 +00:00
Chauncey 6da0078523 [Feat] Allow custom naming of vLLM processes (#21445) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-24 03:15:23 -07:00
Michael Goin 82ec66f514 [V0 Deprecation] Remove Prompt Adapters (#20588) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-23 16:36:48 -07:00
wangxiyuan 1e9438e0b0 [MISC] Move bind_kv_cache to worker module (#20900) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-07-14 09:40:00 +00:00
Nick Hill 657f2f301a [DP] Support external DP Load Balancer mode (#19790) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-02 10:21:52 -07:00
Chauncey d77f7fb871 [Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer (#19283) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-06-08 08:16:31 +08:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Rui Qiao bdce64f236 [V1] Support DP with Ray (#18779) 2025-06-02 21:15:13 -07:00
Nick Hill 2dbe8c0774 [Perf] API-server scaleout with many-to-many server-engine comms (#17546) 2025-05-30 08:17:00 -07:00
Nick Hill 55aa7af994 [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-05-13 10:48:21 -07:00
Daniel Li 48cb2109b6 [V1] Move usage stats to worker and start logging TPU hardware (#16211) 2025-04-25 14:06:01 -06:00
Michael GoinandNick Hill ed50f46641 [Bugfix] Enable V1 usage stats (#16986) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-04-23 19:54:00 -07:00