[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
This commit is contained in:
aoshen02
2026-10-09 08:29:55 +00:00
committed by GitHub
co-authored by Claude Fable 5.1 indianspeedster
parent bacbbe1878
commit a6f43e57d6
9 changed files with 84 additions and 38 deletions
+1 -1
View File
@@ -107,7 +107,7 @@ By default, CUDA graph memory stays on the GPU while asleep. With
`sleep_mode_offload_cudagraph=True` (off by default), CUDA graphs are captured
into a cuMem pool that sleep backs up to CPU memory at both levels and any wake
restores in place, so graphs are replayed, not recaptured. It needs the default
`cumem` backend, CUDA and CUDA graphs; otherwise it has no effect.
`cumem` backend, CUDA or ROCm, and CUDA graphs; otherwise it has no effect.
```python
llm = LLM("Qwen/Qwen3-8B", enable_sleep_mode=True, sleep_mode_offload_cudagraph=True)