[Bugfix][MRV2] Match fast-prefill padding to active LoRA batches (#56456)
Signed-off-by: waizuichougou <2082431897@qq.com> Signed-off-by: waizuichougou <105572299+waizuichougou@users.noreply.github.com>
This commit is contained in:
@@ -149,7 +149,9 @@ class MMEncoderModelRunner(GPUModelRunner):
|
||||
num_tokens=batch_req_state.num_tokens,
|
||||
num_reqs=None,
|
||||
)
|
||||
self.prepare_inputs(scheduler_output, batch_req_state, batch_desc)
|
||||
self.prepare_inputs(
|
||||
scheduler_output, batch_req_state, batch_desc, num_active_loras=0
|
||||
)
|
||||
|
||||
scheduled_encoder_inputs = scheduler_output.scheduled_encoder_inputs
|
||||
if self.lora_config is not None:
|
||||
|
||||
Reference in New Issue
Block a user