[ROCm] Disable AITER FP8/FP4 BMM for MLA by default (#53123)

Signed-off-by: xuebwang-amd <xuebwang@amd.com>
This commit is contained in:
xuebwang
2026-10-14 10:46:02 +00:00
committed by GitHub
parent c67c69e657
commit 81503550f3
4 changed files with 16 additions and 10 deletions
@@ -709,6 +709,11 @@ class MLAAttention(nn.Module, AttentionLayerBase):
use_pcp=self.use_pcp,
)
# TODO(ROCm): both BMM kernels re-quantize kv_b_proj online, ignoring
# the precision the checkpoint asked for, so they now default to off.
# Turning them back on needs long-context accuracy validation, and the
# per-layer decision belongs in --quantization-config.targets rather
# than in this layer.
self.is_aiter_triton_fp8_bmm_enabled = rocm_aiter_ops.is_fp8bmm_enabled()
# If kv_b_proj_weight is unquantized, quantize it to mxfp4 if supported