Pad to 64 not 128 Q heads in flashMLA sparse (#60029)

Signed-off-by: Patryk Saffer <patryk.saffer@mistral.ai>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
This commit is contained in:
PatrykSaffer
2026-10-06 16:03:05 -04:00
committed by GitHub
co-authored by Misha Goin
parent d1f3d8b870
commit e43db1f5e2
2 changed files with 8 additions and 10 deletions
+3 -3
View File
@@ -228,9 +228,9 @@ class HYV4FlashMLASparseImpl(FlashMLASparseImpl):
-1, 1, kv_c_and_k_pe_cache.shape[-1]
)
# NOTE(Chen): kernel requires num_local_head to be a multiple of
# 64 on hopper and 128 on blackwell. Pad from q's head count, not
# self.num_heads: under DCP the heads are all-gathered before this.
# NOTE(Chen): kernel requires num_local_head to be a multiple of 64.
# Pad from q's head count, not self.num_heads: under DCP the heads are
# all-gathered before this.
if actual_num_heads is None:
actual_num_heads = q.shape[1]
padded_num_heads = (