iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,722 Commits 3 Branches 0 Tags
382970ee6ca490aeaaaf4e32c53695b581ff61ba
Commit Graph
30 Commits
Author SHA1 Message Date
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
aoshen02aoshen02Claude Fable 5
 		4ac452ad98 [Core] Release NCCL communicator memory in sleep mode (#51485) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-09-01 14:31:16 +08:00
Dakai AnAlanFokCoOpenAI Codex
 		9409f59e09 [Core] Add CuMemAllocator.discard() for tag-selective GPU memory release (#52514) 
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-16 23:15:14 +08:00
aoshen02andaoshen02 9e50e1037e [Bugfix][CuMem] Make KV-cache wake cleanup tag-safe (#49857) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
 2026-07-26 12:09:13 -07:00
Nils Matteson a461070d1c [Core] Make sleep-mode backend capability flags communicator-agnostic (#47243) 2026-07-01 07:17:44 +00:00
Nils Matteson 93d8f834dd [Core] Pluggable sleep-mode backend abstraction (RFC #34303) (#44074) 2026-06-30 22:00:53 -07:00
peizhang56CursorMatthew WongAndreas Karatzas
 		68afd78897 [Bugfix][ROCm] Fix cumem sleep and teardown (#46203) 
Signed-off-by: pei.zhang <pei.zhang@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-24 02:45:31 +08:00
Nick Hill 7df3d7dada [Core] Ensure memory is pinned prior to async h2d copy (#45424) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-20 20:02:24 -07:00
Yan Ma 3bb46975bd [XPU][Feature] transparent sleep mode support for XPU platform (#37149) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-06-07 10:45:31 +08:00
Harry Mellor a80af24356 Speed up docs build (#44635) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-05 14:51:44 +00:00
qizixiandClaude 18a27cc9a3 [Bugfix] Make CuMemAllocator free callback stream-aware (#43020) 
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-22 03:36:22 +00:00
youkaichaoClaudegemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		2ce95a761b Auto-disable expandable_segments around cumem memory pool (#40812) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-04-27 09:37:22 +08:00
Martin HickeyandHarry Mellor 7560d674c9 [CI] Fix mypy for vllm/device allocator (#35518) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-02 15:53:18 +00:00
tjp_zjuandCyrus Leung 55caa6051d refactor: find_loaded_library (#31866) 
Signed-off-by: tjp_zju <tanjianpingzju1990@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-01-07 06:42:20 +00:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟tjtanaakliuae
 		4ca5cd5740 [Core][AMD] Migrate fully transparent sleep mode to ROCm platform (#12695) 
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
 2025-11-12 15:24:12 -08:00
Jonathan Chen ca76486a16 [Chore] Separate out vllm.utils.platform_utils.py (#27374) 
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
 2025-10-23 19:08:06 +00:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
youkaichao fdb09c77d6 [sleep mode] save memory for on-the-fly quantization (#24731) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-09-12 11:25:19 +00:00
22quinnEric Marcusyoukaichao
 		9dc30b7068 [Bugfix] Add strong reference to CUDA pluggable allocator callbacks (#23477) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Eric Marcus <eric.marcus@kaiko.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-08-24 12:56:17 +08:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Harry Mellor dc372b9c8a Update deprecated type hinting in vllm/device_allocator and vllm/distributed (#18126) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-14 04:07:57 -07:00
Eric Tang ddb94c2605 [core] Add tags parameter to wake_up() (#15500) 
Signed-off-by: Eric <erictang000@gmail.com>
 2025-04-02 01:59:27 -07:00
Lionel Villard 4a9ce1784c [sleep mode] clear pytorch cache after sleep (#15248) 
Signed-off-by: <villard@us.ibm.com>
 2025-03-31 22:58:58 -07:00
youkaichao 71c4b40562 [sleep mode] error out with expandable_segments (#14189) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-03-04 18:54:19 +08:00
youkaichao ac19b519ed [core] fix sleep mode in pytorch 2.6 (#13456) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-02-18 13:48:10 +08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
youkaichao 68ad4e3a8d [Core] Support fully transparent sleep mode (#11743) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-22 14:39:32 +08:00