[Attention] Enable FA4 sparse MQA decode on Blackwell (#55866)
Signed-off-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
This commit is contained in:
co-authored by
Lucas Wilkinson
Claude
Lucas Wilkinson
Matthew Bonanni
parent
0838cd6593
commit
31e90ac370
@@ -147,7 +147,7 @@ Priority is **1 = highest** (tried first).
|
||||
|
||||
--8<-- "gen:priority-mla"
|
||||
|
||||
> **\*** For sparse MLA, FP8 KV cache always prefers `FLASHINFER_MLA_SPARSE`. With BF16 KV cache, `FLASHINFER_MLA_SPARSE` is preferred for low query-head counts (<= 16), while `FLASHMLA_SPARSE` is preferred otherwise.
|
||||
> **\*** For sparse MLA, FP8 KV cache always prefers `FLASHINFER_MLA_SPARSE`. With BF16 KV cache, `FLASH_ATTN_MLA_SPARSE_FA4` (then `FLASHINFER_MLA_SPARSE`) is preferred for low query-head counts (<= 16) on SM 10.x, while `FLASHMLA_SPARSE` is preferred otherwise.
|
||||
>
|
||||
> **Note:** ROCm and CPU platforms have their own selection logic. See the platform-specific documentation for details.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user