[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263)

Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
qizixi
2026-09-26 17:01:17 +08:00
committed by GitHub
co-authored by OpenAI Codex Claude Opus 5.5 Nick Hill
parent 3b4566c5cf
commit a4eb3f25d6
24 changed files with 894 additions and 135 deletions
+2 -1
View File
@@ -243,7 +243,8 @@ def dispatch_cg_and_sync_dp(
dp_size: Data-parallel world size. 1 skips all cross-rank work.
dp_rank: This rank's index in the DP group.
max_query_len: Upper bound on per-request query length, for selecting
varlen decode graphs. None means the graph must not constrain it.
varlen decode graphs. None, as for a batch with a prefill, keeps
the batch out of graphs that constrain it.
need_eager: Force `CUDAGraphMode.NONE` instead of dispatching.
num_active_loras: Active LoRA count for this rank. Does not need
cross-rank agreement; it never changes a bucket's token count.