[Bugfix][PCP] Align MoE routing padding with gathered tokens (#60946)

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Yicong Wang
2026-10-14 21:04:00 -04:00
committed by GitHub
co-authored by Lucas Wilkinson Claude Opus 5.5 OpenAI Codex
parent 8530aed2b7
commit bfdbfa1632
7 changed files with 180 additions and 26 deletions
@@ -376,8 +376,9 @@ class TargetDependentARSpeculator(DraftModelSpeculator):
input_buffers: InputBatch | InputBuffers = self.input_buffers
is_padding = None
if self.pcp_manager is not None:
input_buffers = self.pcp_manager.get_draft_input_buffers(self.input_buffers)
is_padding = input_buffers.is_padding[:num_tokens]
input_buffers, is_padding = self.pcp_manager.get_draft_input_buffers(
self.input_buffers, num_tokens
)
batch_descriptor = BatchDescriptor(num_tokens=num_tokens)
with set_forward_context(
attn_metadata,