[ROCm][DSV4] B-preshuffle the attention fp8 projections (#46720)

Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
Mehmet Cagri
2026-07-30 13:59:52 -05:00
committed by GitHub
co-authored by Claude Opus 4.8 Andreas Karatzas
parent 5f8f72866c
commit 12a34a6bc7
5 changed files with 187 additions and 9 deletions
@@ -142,6 +142,10 @@ def process_weights_after_loading(
if isinstance(module, HpcModule):
module.process_weights_after_loading(model)
# Model-level post-load hook, after the per-layer quant finalize.
if hasattr(model, "process_weights_after_loading"):
model.process_weights_after_loading()
# Needed for torchao model reloading via model.reload_weights
# @kylesayrs @jerryzh168 this can be removed if callers move to `reload_weights`
if model_config.quantization == "torchao":