 Woosuk KwonandClaude Opus 5.5
|
4a89316efa
|
[Core] Drop the Ray driver-node GPU pin left over from the legacy executor (#61713)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 16:00:18 -07:00 |
|
|
Nick Hill
|
c059bee812
|
[MRV2] Switch CI tests still running on MRV1 unnecessarily (#61014)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-10-11 09:57:07 -07:00 |
|
|
|
a6f43e57d6
|
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
|
2026-10-09 08:29:55 +00:00 |
|
|
MattandClaude Opus 5.5
|
5dd4a628dd
|
[ROCm][Bugfix] Map sleep-mode cuMem memory for the host (#60367)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-08 04:26:12 +00:00 |
|
|
aoshen02andClaude Opus 5.5
|
5e56e9af2a
|
[Feature] Release the CUDA graph pool on sleep (#59160)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-04 01:13:10 +00:00 |
|
|
aoshen02andClaude Opus 5.5
|
8e58ac22af
|
[Feature] Offload KV-init runtime state on sleep (#59158)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-02 09:49:21 +00:00 |
|
|
aoshen02andClaude Opus 5.5
|
866f02db8a
|
[Feature] Release WorkspaceManager scratch on sleep (#59156)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-02 02:14:37 +00:00 |
|
|
|
f4917dadc8
|
[Bugfix][Quantization] Stop sleep(level=2) from zeroing compressed-tensors KV scales (#57163)
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-09-29 15:34:15 +00:00 |
|
|
|
da1aaec31f
|
[CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-25 17:05:33 -07:00 |
|
|
|
4013f3ad6c
|
[ROCm] Fix CI runtime and tests for MI355 DPX (#58244)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-09-24 23:15:11 -05:00 |
|
|
Yan MaandKunshang Ji
|
4991f97669
|
[XPU] sleep mode: fix KV cache release test (#57485)
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-09-18 15:49:17 +00:00 |
|
|
Cyrus Leung
|
dbf4b89cf2
|
[Misc] Remove unnecessary Transformers version guards (#57556)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-18 15:01:35 +00:00 |
|
|
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
Oxana KorzhandClaude Opus 5
|
6cddad414e
|
[CI][Test] Deflake test_mem.py sleep-mode asserts via allocator bookeeping (#54312)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-29 12:05:17 -07:00 |
|
|
Ronaldandaoshen02
|
16d6c376bc
|
[Bugfix][Models] Register sleep-managed runtime buffers (#53507)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-08-29 10:32:24 +08:00 |
|
|
Ronaldandaoshen02
|
479eeb32d2
|
[Bugfix][MRV2] Isolate sleep-mode KV allocations (#53508)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-08-27 15:08:28 -07:00 |
|
|
|
61d4f56635
|
[Offloader] Offload submodules that make_layers never reaches (#53120)
Signed-off-by: ray24777 <103923677+ray24777@users.noreply.github.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-26 18:37:00 +08:00 |
|
|
|
9409f59e09
|
[Core] Add CuMemAllocator.discard() for tag-selective GPU memory release (#52514)
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-16 23:15:14 +08:00 |
|
|
Nick Hill
|
9b0afeb4f6
|
[Perf] Avoid some more unnecessary GPU<->CPU syncs (#51458)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-08 17:13:26 -07:00 |
|
|
Wentao Ye
|
c39076feff
|
[CI Test] Add specific unit test for mrv2 offloading (#51440)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-07 22:49:45 +00:00 |
|
|
Wentao Ye
|
a671679e9f
|
[MRv2 Feature] MR v2 weight offloading support (#51413)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-07 13:20:21 -06:00 |
|
|
|
b50fdebce0
|
[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True (#39935)
Signed-off-by: Silen Naihin <silen.naihin@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-08-06 09:13:10 +08:00 |
|
|
Tyler Michael SmithandOpenAI Codex
|
12292d94b2
|
[CI] Detect and fail evals on when NaNs appear in logits (#50323)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-04 23:02:58 +00:00 |
|
|
Andreas Karatzas
|
165ed33327
|
[CI][ROCm] Stabilize LLM GC teardown check (#50340)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-07-30 18:42:03 +08:00 |
|
|
Andreas Karatzas
|
3ad85e0de4
|
[CI][AMD] Configure MI300 tests for native execution without DinD (#48387)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-15 02:14:16 +00:00 |
|
|
Kunshang Ji
|
fb42e5219e
|
[Platform] Replace torch.cuda.mem_get_info with torch.accelerator.get_memory_info (#44825)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
|
2026-06-30 14:39:52 +08:00 |
|
|
|
3f5a1e1733
|
[ROCm][CI] Expand basic correctness target suites (#46573)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Matt <156021403+mawong-amd@users.noreply.github.com>
|
2026-06-25 12:18:57 +08:00 |
|
|
Yan Ma
|
3bb46975bd
|
[XPU][Feature] transparent sleep mode support for XPU platform (#37149)
Signed-off-by: Yan Ma <yan.ma@intel.com>
|
2026-06-07 10:45:31 +08:00 |
|
|
wliao2
|
3abf858443
|
[Test] Refactor hard coded device string in test files under compile/quantization/models/model_executor folders (#38901)
Signed-off-by: Liao, Wei <wei.liao@intel.com>
|
2026-04-15 11:02:35 +08:00 |
|
|
Harry Mellor
|
65986db6ba
|
Make Gemma and Gemma 2 accept inputs_embeds like Gemma 3 (#36787)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-03-11 18:12:43 +00:00 |
|
|
|
6f0dd93801
|
[Core] Remove busy loop from idle buffer readers (#28053)
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-03-04 07:44:20 +00:00 |
|
|
Ming YangandMichael Goin
|
6831650c40
|
[offloader] v2: Hide weight onloading latency via prefetching (#29941)
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-02-25 17:20:59 -08:00 |
|
|
Wei ZhaoandMichael Goin
|
59d53066d8
|
[Feature] Support CPU Offloading without Pytorch Pinned Memory that leads to doubled allocation (#32993)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-02-13 08:11:26 -08:00 |
|
|
Ning Xie
|
c907d22158
|
[refactor] refactor memory constants usage (#31865)
Signed-off-by: Andy Xie <andy.xning@gmail.com>
|
2026-01-07 18:37:31 +00:00 |
|
|
Matthew Bonanni
|
7eb6cb6c18
|
[Attention] Update tests to remove deprecated env vars (#30563)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2025-12-17 09:49:59 -08:00 |
|
|
Charlie Fu
|
6af70e11a0
|
[ROCm][CI] Fix test_max_len.py for Rocm (#29916)
Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Charlie Fu <Charlie.Fu@amd.com>
|
2025-12-08 16:58:30 -05:00 |
|
|
Charlie Fu
|
2c22c4ca2d
|
[ROCm][CI] Increase the memory threshold for test_deep_sleep_fp8_kvcache (#30104)
Signed-off-by: charlifu <charlifu@amd.com>
|
2025-12-05 04:51:44 +00:00 |
|
|
VensenandTJian
|
66b5840287
|
[Bugfix][sleepmode][fp8 kv cache]: Fix FP8 KV cache + sleep(level=2) gibberish output (#28783)
Signed-off-by: vensen <vensenmu@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2025-11-30 14:24:25 +08:00 |
|
|
Roger Wang
|
0ff70821c9
|
[Core] Deprecate xformers (#29262)
Signed-off-by: Roger Wang <hey@rogerw.io>
|
2025-11-24 04:18:55 +00:00 |
|
|
|
4ca5cd5740
|
[Core][AMD] Migrate fully transparent sleep mode to ROCm platform (#12695)
Signed-off-by: Hollow Man <hollowman@opensuse.org>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
|
2025-11-12 15:24:12 -08:00 |
|
|
Sumanth R Hegde
|
4917002523
|
[Fix] Skip record_sleep_state logic in PrometheusStatsLogger if not in dev mode (#27789)
Signed-off-by: SumanthRH <sumanthrh99@gmail.com>
|
2025-10-30 19:26:27 +00:00 |
|
|
Nick Hill
|
647214f3d5
|
[V0 Deprecation] Remove V0 executors (#27142)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-10-21 11:09:37 -07:00 |
|
|
|
1d165d6d85
|
[Chore] Separate out vllm.utils.mem_utils (#27143)
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-10-18 10:06:59 +00:00 |
|
|
|
43721bc67f
|
[CI] Replace large models with tiny alternatives in tests (#24057)
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-10-16 15:51:27 +01:00 |
|
|
Cyrus Leung
|
1e4ecca1d0
|
[V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-10-07 15:42:31 +00:00 |
|
|
Harry Mellor
|
d6953beb91
|
Convert formatting to use ruff instead of yapf + isort (#26247)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-10-05 07:06:22 -07:00 |
|
|
Woosuk Kwon
|
52c2a8d4ad
|
[V0 Deprecation] Remove LLMEngine (#25033)
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-09-20 17:56:30 -07:00 |
|
|
Andrew SansomandCyrus Leung
|
9a4600e4dc
|
[CORE] Prompt Embeddings Support for v1 Engine (#24278)
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2025-09-19 08:03:09 +08:00 |
|