25 lines
672 B
Markdown
25 lines
672 B
Markdown
# B12X Linear Backend
|
|
|
|
[B12X](https://pypi.org/project/b12x/) provides optional CUDA kernels for
|
|
NVIDIA SM120 and SM121 GPUs. Install the dependency with:
|
|
|
|
```bash
|
|
uv pip install "vllm[b12x]"
|
|
```
|
|
|
|
B12X participates in automatic kernel selection after established optimized
|
|
backends and before emulation. Select it explicitly with:
|
|
|
|
```bash
|
|
vllm serve <model> --linear-backend b12x
|
|
```
|
|
|
|
## Supported Configurations
|
|
|
|
| Backend | Supported configurations |
|
|
| ------- | ------------------------ |
|
|
| Linear | Per-tensor FP8, 128x128 block FP8, MXFP8, NVFP4, and MXFP4 |
|
|
|
|
Dense W4A16 layers are not handled by B12X and continue to use another
|
|
compatible backend such as Marlin.
|