[Core][Spec Decode] Replace EAGLE group heuristics with explicit draft-layer tags (#61455)

Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Dakai An
2026-10-17 08:22:33 +00:00
committed by GitHub
co-authored by Yifan Qiao Claude Opus 5.5
parent 08a65e287e
commit c3de0b37e2
10 changed files with 160 additions and 354 deletions
@@ -236,10 +236,9 @@ class SchedulerOffloadConfig(NamedTuple):
)
if use_eagle_block_drop and not eagle_groups:
# No group is annotated as holding drafter layers. This happens
# for shared-group MTP models (e.g. Qwen3.5-style), whose drafter
# is a regular decoder layer merged into the target's
# full-attention group, and for separate-draft models that miss
# annotation. Treating every group as a drafter group here makes
# when the drafter owns no KV cache (e.g. KV-sharing drafters) or
# the model runner does not tag draft layers' KV cache specs.
# Treating every group as a drafter group here makes
# the volatile-tail pop apply to ALL groups, which can zero the
# whole request's offload hit whenever any group's servable
# window is a single chunk (issue #52735). Drafter KV served one