[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm (#59523)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: indianspeedster <cspandey016@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: indianspeedster <cspandey016@gmail.com>
This commit is contained in:
aoshen02
2026-10-09 08:29:55 +00:00
committed by GitHub
co-authored by Claude Fable 5.1 indianspeedster
parent bacbbe1878
commit a6f43e57d6
9 changed files with 84 additions and 38 deletions
-22
View File
@@ -226,28 +226,6 @@ void create_and_map(unsigned long long device, ssize_t size, CUdeviceptr d_mem,
return;
}
#ifdef USE_ROCM
// ROCm-only: zero the freshly (re)mapped region.
//
// On some amdgpu driver versions the physical VRAM backing a buffer is not
// scrubbed on release, so the pages handed back here (e.g. on wake_up after a
// sleep that discarded a tag) can still contain stale data from a previously
// freed allocation -- possibly from another process. Callers of the device
// allocator expect newly mapped memory to be well defined; in particular
// hybrid Mamba / gated-delta-net (GDN) models keep a persisted conv/recurrent
// state cache, and a new sequence that reads an unzeroed slot will propagate
// garbage/NaN, collapsing generation to token id 0 (decoded as "!").
//
// This is scoped to the ROCm path on purpose: the CUDA path is unchanged.
// Use CUDA_CHECK so a failure sets the global error_code; the Python caller
// (python_create_and_map) only inspects error_code, so returning without
// setting it would make a failed memset look successful and still hand back
// unzeroed memory.
CUDA_CHECK((CUresult)hipMemset(reinterpret_cast<void*>(d_mem), 0, size));
if (error_code != 0) {
return;
}
#endif
// std::cout << "create_and_map: device=" << device << ", size=" << size << ",
// d_mem=" << d_mem << ", p_memHandle=" << p_memHandle << std::endl;
}