[Bugfix] Include inline per-token-head scales in offloaded page transfer width (#48411)

Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Itay Etelis <Itay.etelis@gmail.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
This commit is contained in:
Itay Etelis
2026-07-14 16:07:26 +03:00
committed by GitHub
co-authored by Itay Etelis Itay Etelis
parent 7a74a9662b
commit 32aef44388
3 changed files with 41 additions and 6 deletions
@@ -106,7 +106,7 @@ class OffloadingConnectorWorker:
)
page_size_bytes[layer_name] = layer_kv_cache_spec.page_size_bytes
unpadded_page_size_bytes[layer_name] = (
layer_kv_cache_spec.real_page_size_bytes
layer_kv_cache_spec.unpadded_page_size_bytes
)
elif isinstance(layer_kv_cache_spec, MambaSpec):