[Bugfix][MRV2] Use int64 index mappings to fix row-offset overflow (#60736)

Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Nick Hill
2026-10-09 14:38:11 -07:00
committed by GitHub
co-authored by vllmellm Claude Opus 5.5
parent 51144dd83e
commit ed7ae705a1
10 changed files with 221 additions and 24 deletions
@@ -102,7 +102,7 @@ class DFlashSpeculator(DraftModelSpeculator):
# full buffer before a real batch has populated it, so zero would make
# every padding row scatter into request slot 0.
self.sample_idx_mapping = torch.full(
(max_num_sampled_tokens,), -1, dtype=torch.int32, device=device
(max_num_sampled_tokens,), -1, dtype=torch.int64, device=device
)
# [0, 1, ..., N-1, 0, 1, ..., N-1, ...] -> the per-token column index into
# draft_logits[req, step, :].