[Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233)
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com> Signed-off-by: Ido Asraff <idoa@atero.ai>
This commit is contained in:
@@ -920,7 +920,7 @@ def get_kv_transfer_group() -> kv_transfer.KVTransferAgent:
|
||||
|
||||
|
||||
@contextmanager
|
||||
def graph_capture():
|
||||
def graph_capture(device: torch.device):
|
||||
"""
|
||||
`graph_capture` is a context manager which should surround the code that
|
||||
is capturing the CUDA graph. Its main purpose is to ensure that the
|
||||
@@ -934,8 +934,9 @@ def graph_capture():
|
||||
in order to explicitly distinguish the kernels to capture
|
||||
from other kernels possibly launched on background in the default stream.
|
||||
"""
|
||||
with get_tp_group().graph_capture() as context, get_pp_group(
|
||||
).graph_capture(context):
|
||||
context = GraphCaptureContext(torch.cuda.Stream(device=device))
|
||||
with get_tp_group().graph_capture(context), get_pp_group().graph_capture(
|
||||
context):
|
||||
yield context
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user