[Hardware][XPU] Register matmul and linear batch-invariant kernels for XPU (#49209)
Signed-off-by: tzielinski-habana <tomasz.zielinski@intel.com> Signed-off-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
co-authored by
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Claude Opus 5
Harry Mellor
parent
7894394b07
commit
ffe3bb3c72
@@ -17,7 +17,29 @@ Batch invariance is crucial for several use cases:
|
||||
|
||||
## Hardware Requirements
|
||||
|
||||
Batch invariance requires NVIDIA GPUs with compute capability 8.0 or higher.
|
||||
Batch invariance is supported on the following platforms:
|
||||
|
||||
- NVIDIA GPUs with compute capability 8.0 or higher.
|
||||
- Intel XPUs with Triton support.
|
||||
|
||||
### Attention Backend Selection for XPU
|
||||
|
||||
On XPU, Triton Attention backend is required for batch invariance.
|
||||
Select this backend using Qwen/Qwen3-1.7B model as an example:
|
||||
|
||||
```python
|
||||
llm = LLM(
|
||||
model="Qwen/Qwen3-1.7B",
|
||||
attention_config={"backend": "TRITON_ATTN"},
|
||||
)
|
||||
```
|
||||
|
||||
Or via the CLI:
|
||||
|
||||
```bash
|
||||
VLLM_BATCH_INVARIANT=1 vllm serve Qwen/Qwen3-1.7B \
|
||||
--attention-config.backend TRITON_ATTN
|
||||
```
|
||||
|
||||
## Enabling Batch Invariance
|
||||
|
||||
|
||||
Reference in New Issue
Block a user