[Bugfix][Model Runner V2] Order uniform decodes first so spec decodes aren't misclassified as prefills (#47381)

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
Woosuk Kwon
2026-07-09 14:26:27 +01:00
committed by GitHub
co-authored by Claude Fable 5 Nick Hill
parent b83be00cdd
commit 85b3a7264b
3 changed files with 86 additions and 3 deletions
+6 -1
View File
@@ -1156,7 +1156,12 @@ class DeepseekV2MLAAttention(nn.Module):
cache_config,
quant_config,
prefix,
skip_topk=_skip_topk,
# MTP layers must never start with skip_topk=True: their indexer
# computes indices at draft step 0, and the runtime toggle
# (set_skip_topk, index_share_for_mtp_iteration) only exists in
# the V1 proposer. A frozen True would leave the draft reading a
# never-written topk buffer.
skip_topk=_skip_topk and not is_mtp_layer,
)
def forward(