[Kernel] Add B12X causal paged attention backend (#52017)
Signed-off-by: Luke Alonso <lalonso@gmail.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
This commit is contained in:
co-authored by
Lucas Wilkinson
parent
adebc41b7e
commit
0d4ad47981
@@ -136,6 +136,16 @@ Priority is **1 = highest** (tried first).
|
||||
> `--block-size` that is not a multiple of 128 instead makes FlashAttention ineligible
|
||||
> for such models, which is an error if the backend was requested explicitly.
|
||||
|
||||
### b12x
|
||||
|
||||
The optional [b12x](https://pypi.org/project/b12x/) backend supports causal
|
||||
decoder attention on NVIDIA SM120 and SM121 GPUs. Install and select it with:
|
||||
|
||||
```bash
|
||||
uv pip install "vllm[b12x]"
|
||||
vllm serve <model> --attention-backend b12x
|
||||
```
|
||||
|
||||
## MiniMax M3 Sparse Attention Backends
|
||||
|
||||
Block-sparse GQA backend used by MiniMax M3 sparse ("lightning indexer")
|
||||
|
||||
Reference in New Issue
Block a user