[Bugfix] OffloadingConnector: respect skip_reading_prefix_cache flag (#44592)

Signed-off-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Co-authored-by: Or Ozeri <or@ozery.com>
This commit is contained in:
littlecircle0730
2026-06-12 02:20:39 +00:00
committed by GitHub
co-authored by Hsiao-Yuan Chen Or Ozeri
parent e0871ad225
commit 4bc83323f2
3 changed files with 61 additions and 2 deletions
@@ -571,7 +571,11 @@ class OffloadingConnectorScheduler:
req_status.update_offload_keys()
req_status.num_locally_computed_tokens = num_computed_tokens
num_hit_tokens = self._lookup(req_status)
num_hit_tokens: int | None
if request.skip_reading_prefix_cache:
num_hit_tokens = 0
else:
num_hit_tokens = self._lookup(req_status)
req_status.update_num_hit_blocks(num_computed_tokens + (num_hit_tokens or 0))
self._touch(req_status)