[Bug] Fix rocm sparse attn indexer issue (#39225)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
This commit is contained in:
Wentao Ye
2026-04-13 07:53:45 -07:00
committed by GitHub
co-authored by TJian
parent d8ddb31644
commit ccd0d1d906
@@ -532,6 +532,11 @@ def rocm_aiter_sparse_attn_indexer(
has_prefill = layer_attn_metadata.num_prefills > 0
num_decode_tokens = layer_attn_metadata.num_decode_tokens
# during speculative decoding, k may be padded to the CUDA graph batch
# size while slot_mapping only covers actual tokens.
num_tokens = slot_mapping.shape[0]
k = k[:num_tokens]
ops.indexer_k_quant_and_cache(
k,
kv_cache,