[Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks (#49052)

Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: jasl <jasl9187@hotmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
This commit is contained in:
coltonottley
2026-07-26 14:53:32 +03:00
committed by GitHub
co-authored by Colton Ottley jasl Claude Opus 4.8 Or Ozeri
parent da3a252fd1
commit 5559679229
@@ -860,6 +860,7 @@ class OffloadingConnectorScheduler:
num_pending_gpu_blocks
<= group_config.sliding_window_size_in_chunks
* self.config.blocks_per_chunk
+ 1
)
num_chunks = cdiv(num_cached_tokens, tokens_per_chunk)