[Bugfix] Prevent padding placeholders from reaching embeddings (#47029)

Signed-off-by: qianlihuang <91178480+qianlihuang@users.noreply.github.com>
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
Yiliu Dong
2026-07-01 09:26:03 -07:00
committed by GitHub
co-authored by Nick Hill
parent c8bdcc0116
commit 00eb7cefa3
3 changed files with 60 additions and 0 deletions
+4
View File
@@ -3440,6 +3440,10 @@ class GPUModelRunner(
is_first_rank = get_pp_group().is_first_rank
is_encoder_decoder = self.model_config.is_encoder_decoder
# Clamp speculative scheduler placeholders (-1) before embedding lookup.
if self.speculative_config is not None:
self.input_ids.gpu[:num_input_tokens].clamp_(min=0)
# _prepare_inputs may reorder the batch, so we must gather multi
# modal outputs after that to ensure the correct order
ec_connector_output = None