[Attention] Integrate FlashAttention 4 SM100 headdim 256 support (#42669)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
parent
8112b6c997
commit
bf2b45b5d6
@@ -153,7 +153,9 @@ hardware and configuration.
|
||||
|
||||
> **‡** Automatic selection tries FlashAttention first. On Blackwell
|
||||
> (SM100), the fallback order is TRT-LLM Ragged, FlashInfer, then
|
||||
> TokenSpeed MLA. On other GPUs, only FlashAttention is considered.
|
||||
> TokenSpeed MLA; for (qk_nope_head_dim=192, qk_rope_head_dim=64,
|
||||
> v_head_dim=256) TRT-LLM Ragged is tried before FlashAttention. On other
|
||||
> GPUs, only FlashAttention is considered.
|
||||
|
||||
### Decode Backends
|
||||
|
||||
|
||||
Reference in New Issue
Block a user