[Bugfix] Don't split a multimodal item that a prefill chunk starts exactly on (#52412)

Signed-off-by: Taksh <takshkothari09@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
Taksh Kothari
2026-10-13 15:32:38 +00:00
committed by GitHub
parent b9b401b2ef
commit 12b005eefa
2 changed files with 38 additions and 1 deletions
+4 -1
View File
@@ -1868,7 +1868,10 @@ class Scheduler(SchedulerInterface):
# only cover part of the mm input, roll back to before the mm item.
if (
self.scheduler_config.disable_chunked_mm_input
and num_computed_tokens < start_pos
and not self.is_encoder_decoder
# `<=`, not `<`: a chunk that starts exactly at the item still
# splits it when it cannot cover the whole item.
and num_computed_tokens <= start_pos
and (num_computed_tokens + num_new_tokens)
< (start_pos + num_encoder_tokens)
):