[Bugfix] Make CuMemAllocator free callback stream-aware (#43020)

Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
qizixi
2026-05-22 03:36:22 +00:00
committed by GitHub
co-authored by Claude
parent 0ddd7dd656
commit 18a27cc9a3
+6
View File
@@ -160,6 +160,12 @@ class CuMemAllocator:
data = self.pointer_to_data.pop(ptr)
if data.cpu_backup_tensor is not None:
data.cpu_backup_tensor = None
# Drain pending kernels before the C extension's cuMemUnmap.
# The pluggable allocator path doesn't defer reclaim like the
# regular caching allocator, so without this, in-flight work
# (e.g. quant helpers' transient tensors during weight loading)
# races the unmap and surfaces as CUDA_ERROR_ILLEGAL_ADDRESS.
torch.cuda.synchronize(data.handle[0])
logger.debug(
"Freed %s bytes for %s with address %s from cumem allocator",
data.handle[1],