[Bugfix][MRV2] Keep input_ids on later PP stages for raw-token models (#61016)

Signed-off-by: booth-algo <kevinlauofficial01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
This commit is contained in:
Kevin
2026-10-15 12:54:21 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 Kevin H. Luu
parent d486219fc2
commit d12ea8e309
6 changed files with 51 additions and 5 deletions
+2 -1
View File
@@ -1970,7 +1970,8 @@ class GPUModelRunner(LoRAModelRunnerMixin):
}
if not self.is_first_pp_rank:
# Update for non-first PP ranks.
model_inputs["input_ids"] = None
if not requires_raw_input_tokens(self.model):
model_inputs["input_ids"] = None
model_inputs["inputs_embeds"] = None
# Prepare the intermediate tensors.