[Bugfix][PD] Pad resumed speculative decode requests (#55126)

Signed-off-by: Ziming Huang <48115868+ZeldaHuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Ziming Huang
2026-09-04 11:05:33 +08:00
committed by GitHub
co-authored by OpenAI Codex
parent 25268f0c9c
commit a26b71d868
2 changed files with 18 additions and 1 deletions
+2 -1
View File
@@ -973,7 +973,8 @@ class Scheduler(SchedulerInterface):
(self.num_spec_tokens > 0 and self.dynamic_sd_lookup is None)
and self.num_sampled_tokens_per_step > 0
and num_new_tokens == 1
and (scheduled_running_reqs and not prefill_scheduled)
and not prefill_scheduled
and (scheduled_running_reqs or num_computed_tokens > 0)
):
padded_num_tokens = 1 + self.num_spec_tokens
# Pad only when there is room for the sampled token(s).