[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263)

Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
qizixi
2026-09-26 17:01:17 +08:00
committed by GitHub
co-authored by OpenAI Codex Claude Opus 5.5 Nick Hill
parent 3b4566c5cf
commit a4eb3f25d6
24 changed files with 894 additions and 135 deletions
+2 -1
View File
@@ -69,7 +69,8 @@ class BatchExecutionDescriptor:
num_reqs: int | None # None means no request padding is needed (PIECEWISE graphs)
uniform_token_count: int | None = None
# Upper bound on per-request query length. Varlen decode graphs leave
# uniform_token_count unset, so this is what keeps a prefill batch out of one.
# uniform_token_count unset, so this is what keeps a prefill batch out of one:
# the runner passes None for any batch with a prefill.
max_query_len: int | None = None
num_active_loras: int = 0
# Number of microbatches the batch is split into (DBO). 1 means no splitting.