[Perf] Pin CPU tensors before non_blocking H2D in three MM paths (#54292)

Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Keyi Li
2026-08-28 20:13:40 -07:00
committed by GitHub
co-authored by Keyi Li Claude Opus 4.7
parent 93ab92be0c
commit 46a83642f6
3 changed files with 5 additions and 11 deletions
@@ -259,10 +259,7 @@ class MMEncoderAttention(CustomOp):
sequence_lengths = add_padding_to_seqlens(
sequence_lengths, len(sequence_lengths), 0
)
sequence_lengths = torch.from_numpy(sequence_lengths).to(
device, non_blocking=True
)
return sequence_lengths
return async_tensor_h2d(sequence_lengths, device=device)
@classmethod
def maybe_recompute_cu_seqlens(