[Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata (#36626)

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
This commit is contained in:
Woosuk Kwon
2026-03-10 09:30:56 -07:00
committed by GitHub
parent f83b933b84
commit f088a831dd
4 changed files with 14 additions and 3 deletions
+1
View File
@@ -384,6 +384,7 @@ def prepare_inputs_to_capture(
attn_metadata = model_state.prepare_attn(
input_batch,
CUDAGraphMode.NONE,
input_block_tables,
slot_mappings,
attn_groups,