[Bugfix][Core] Schedule encoder-only prompts larger than one step (#59029)
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io> Co-authored-by: Isotr0py <2037008807@qq.com>
This commit is contained in:
co-authored by
Isotr0py
parent
02a3c6dfc5
commit
2e52a558f0
@@ -1115,6 +1115,7 @@ class Scheduler(SchedulerInterface):
|
||||
if (
|
||||
not self.scheduler_config.enable_chunked_prefill
|
||||
and num_new_tokens > request_token_budget
|
||||
and not self.is_mm_encoder_only
|
||||
):
|
||||
# If chunked_prefill is disabled,
|
||||
# we can stop the scheduling here.
|
||||
|
||||
Reference in New Issue
Block a user