[ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes (#58167)

Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Simon Danielsson
2026-10-02 11:47:41 +01:00
committed by GitHub
co-authored by mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 751fa96a49
commit 98b29aa99a
8 changed files with 705 additions and 240 deletions
+3
View File
@@ -167,6 +167,7 @@ SparseIndexerTopkBackend = Literal[
"per_row",
"flashinfer",
"torch",
"aiter",
]
# Architectures whose model code wires up the flashinfer moe_ep experts. MTP
@@ -289,6 +290,8 @@ class KernelConfig:
- "per_row": Use vLLM's top_k_per_row_decode kernel
- "flashinfer": Use FlashInfer's top_k_ragged_transform kernel
- "torch": Use a plain torch.topk implementation (debug reference)
- "aiter": Use AITER's top_k_per_row_decode kernel (ROCm gfx950 only,
requires VLLM_ROCM_USE_AITER)
Explicit values raise RuntimeError when their constraints are not met.
"""