[SM100] Hdim 256 optimized (#52980)

Signed-off-by: Simon Veitner <sveitner@redhat.com>
This commit is contained in:
simon-veitner-redhat
2026-08-25 20:28:40 -07:00
committed by GitHub
parent f6130145c8
commit 7156c63bef
7 changed files with 423 additions and 22 deletions
+8
View File
@@ -127,6 +127,14 @@ Priority is **1 = highest** (tried first).
> **†** FlashInfer Native is the regular FlashInfer path. XQA is the SM90 decode path exposed through FlashInfer's TRTLLM decode API. trtllm-gen is used on SM100 and supports sinks. Disable XQA/trtllm-gen via `--attention-config.use_trtllm_attention=0`.
>
> **\*** Specify the FlashAttention version via `--attention-config.flash_attn_version=2`, `3`, or `4`. Default is FA4 on SM100+ (Blackwell), FA3 on SM90 (Hopper), FA2 otherwise.
>
> On Blackwell, when the FlashAttention backend is selected, `head_size=256` is served
> by a dedicated FA4 kernel that requires a KV cache block size of 128 (advertised
> automatically, so it is picked as long as `--block-size` is not pinned) and does not
> support logit soft capping, attention sinks, mm_prefix/R-SWA masking, DCP, or windowed
> encoder attention. Those configurations transparently fall back to FA2. A pinned
> `--block-size` that is not a multiple of 128 instead makes FlashAttention ineligible
> for such models, which is an error if the backend was requested explicitly.
## MiniMax M3 Sparse Attention Backends