[Perf][Kernel] Tune cooperative topk for medium batch-sizes (#53382)

Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Roberto L. Castro
2026-08-31 13:44:38 -07:00
committed by GitHub
co-authored by Codex mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 85c1365bd9
commit f5c3cc240b
4 changed files with 33 additions and 10 deletions
@@ -617,7 +617,7 @@ def sparse_attn_indexer(
use_cooperative_topk = (
current_platform.is_cuda()
and topk_tokens in (512, 1024, 2048)
and num_rows <= 32
and num_rows <= 64
and logits.stride(0) % 4 == 0 # TMA 16-byte alignment
and current_platform.has_device_capability(90)
and not current_platform.is_device_capability_family(120)