[DSA] Wire DeepGEMM sparse MQA logits into the DeepSeek V4.1 indexer (#56254)

Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jared Wen
2026-09-15 14:07:14 -07:00
committed by GitHub
co-authored by Claude Fable 5.1
parent f2aad6aa70
commit 1257512305
13 changed files with 1830 additions and 26 deletions
+9
View File
@@ -86,6 +86,15 @@ class AttentionConfig:
indexer). Quantized formats (fp8, mxfp4, nvfp4) require indexer kernel
support in the backend."""
indexer_sparse_logits: bool = False
"""DeepSeek V4.1 two-level indexer: score only the candidate blocks with
DeepGEMM's sparse MQA-logits kernels instead of computing dense logits over
the whole context and masking them. Requires `indexer_kv_dtype="mxfp4"`,
an SM100-class GPU, DeepGEMM >= 2.8 and the DeepSelect top-k extension
(the top-k runs on the kernels' bf16 logits). The sparse path costs
O(candidate blocks) per query regardless of context length, so it pays off
for long contexts (roughly 32K tokens and beyond) and is slower below."""
hisparse_config: HiSparseConfig | None = None
"""HiSparse host-resident KV configuration. Setting this enables experimental
Model Runner V2-only HiSparse sparse-MLA decode hot-buffering. It is inferred