[Model Runner v2] Fix block table IMA issue (#40648)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
This commit is contained in:
Wentao Ye
2026-04-29 08:30:33 -07:00
committed by GitHub
co-authored by Nick Hill
parent 39a7f4f4e2
commit 51fda1ba44
4 changed files with 30 additions and 22 deletions
+3
View File
@@ -884,6 +884,9 @@ class GPUModelRunner(
self.encoder_cache.clear()
self.late_interaction_runner.clear()
def post_kv_cache_wake_up(self) -> None:
self.init_fp8_kv_scales()
@torch.inference_mode()
def init_fp8_kv_scales(self) -> None:
"""