[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer (#57263)
Signed-off-by: zixi-qi <zixi@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
co-authored by
OpenAI Codex
Claude Opus 5.5
Nick Hill
parent
3b4566c5cf
commit
a4eb3f25d6
@@ -243,7 +243,8 @@ def dispatch_cg_and_sync_dp(
|
||||
dp_size: Data-parallel world size. 1 skips all cross-rank work.
|
||||
dp_rank: This rank's index in the DP group.
|
||||
max_query_len: Upper bound on per-request query length, for selecting
|
||||
varlen decode graphs. None means the graph must not constrain it.
|
||||
varlen decode graphs. None, as for a batch with a prefill, keeps
|
||||
the batch out of graphs that constrain it.
|
||||
need_eager: Force `CUDAGraphMode.NONE` instead of dispatching.
|
||||
num_active_loras: Active LoRA count for this rank. Does not need
|
||||
cross-rank agreement; it never changes a bucket's token count.
|
||||
|
||||
Reference in New Issue
Block a user