[Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291)
Signed-off-by: NickLucche <nlucches@redhat.com>
This commit is contained in:
@@ -1147,6 +1147,7 @@ class Scheduler:
|
||||
|
||||
# Update swapped requests.
|
||||
self.swapped.extend(running_scheduled.swapped_out)
|
||||
# Put prefills first due to Attention backend ordering assumption.
|
||||
return SchedulerOutputs(
|
||||
scheduled_seq_groups=(prefills.seq_groups +
|
||||
running_scheduled.prefill_seq_groups +
|
||||
|
||||
Reference in New Issue
Block a user