 liranschourandKevin H. Luu
|
c5f8adc0b1
|
[KV Offload] Expose OffloadingManager.lock()/unlock() and service the tiering control plane off the scheduler thread (#58168)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 08:50:56 +03:00 |
|
 
|
775fefa804
|
[Metrics][KV Offload] Add vllm:kv_offload_config_info - generic info metrics for offloading the offloading connector (#56867)
Signed-off-by: Amir Friedman <Amir.Friedman1@ibm.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-10-14 12:34:37 +00:00 |
|
  
|
e37e51dd24
|
[Metrics] Expose cached prompt tokens by cache tier (#56318)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-06 14:52:05 -04:00 |
|
 bnellnmandItay Etelis
|
23e26e0588
|
[KV Offloading] Back-pressure detection and remediation (#50045)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
|
2026-09-18 14:47:48 -04:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
 MINJUN GILandOr Ozeri
|
f12fe10c5c
|
[Bugfix][KV Offload] Track cache recency once per request (#51787)
Signed-off-by: mindungil <alswnsrlf12@naver.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-16 10:47:26 +03:00 |
|
Ronen Schaffer
|
138d137b5b
|
[Refactor][kv_offload]: rename block→chunk (#52615)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-09-09 17:45:49 +03:00 |
|
Ilia Yastrebov
|
41848caa63
|
[NIXL] Use int32 array for indices to avoid intermediate conversion (#51952)
Signed-off-by: Ilia Yastrebov <iyastrebov@nvidia.com>
|
2026-09-02 16:30:16 +02:00 |
|
 AlexHuangandOr Ozeri
|
c35551f892
|
[Bugfix][KV Offload] Isolate tiering shutdown failures (#52290)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-01 12:24:54 +03:00 |
|
Almog Tavor
|
bc39ded3c4
|
[Bugfix][KV Offload] Defer request-level cascade of in-flight primary keys (#53329)
Signed-off-by: almogtavor <almogtavor@gmail.com>
|
2026-08-25 23:20:03 +00:00 |
|
 Varun Sundar RabindranathandVarun Sundar Rabindranath
|
f067737b2a
|
[Bugfix][TieredOffloading] : Return HIT_PENDING when KV promotion is triggered (#51840)
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-08-12 05:07:16 +03:00 |
|
 Srinivas Krovvidiandsrinivas_oo7
|
a123159f7a
|
Add tiering offloading metrics (#48798)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-08-10 05:23:14 +03:00 |
|
 
|
41a7e7da0c
|
[KV Offload] Support partial secondary-tier load results (#50321)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-04 20:48:58 -07:00 |
|
Philip Pesic
|
dc1be79031
|
Add CachePolicyFactory for pluggable/external eviction policies (#49114)
Signed-off-by: Philip Pesic <philippesic06@gmail.com>
|
2026-07-29 07:34:55 +03:00 |
|
Ronen Schaffer
|
77cba0259f
|
[KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-07-27 13:29:57 +03:00 |
|
 Srinivas Krovvidiandsrinivas_oo7
|
ce65385618
|
[KV Offload] Split tiering_lookup_delay into sync/async histograms (#47679)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-07-16 20:03:45 +03:00 |
|
 
|
c46ced1ee3
|
[kv_offload] Establish tier-owned KV event handling (#46544)
Signed-off-by: Change72 <changg@nvidia.com>
Signed-off-by: Chang Guo <changg@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 16:55:20 +03:00 |
|
 Ronen SchafferandOr Ozeri
|
48fcfc926c
|
[KV Offload] Add ParentManager ABC for secondary tier callbacks (#47274)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 13:51:18 +03:00 |
|
 Ronen SchafferandOr Ozeri
|
0fc2512094
|
[KV Offload] Pass ScheduleEndContext to on_schedule_end hook (#46450)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-30 17:07:12 +03:00 |
|
 Ronen SchafferandOr Ozeri
|
bb61177e49
|
[KV Offloading] Replace bool|None lookup return with LookupResult enum (#46363)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-24 18:06:08 +03:00 |
|
 Rui YinandOr Ozeri
|
cf9fd6457e
|
Fix KV offload request-finished lifecycle contract (#46284)
Signed-off-by: test test <2260891073@qq.com>
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-24 15:42:40 +03:00 |
|
 Srinivas Krovvidiandsrinivas_oo7
|
091bc1026e
|
[KV Offloading] Add tiering metric plumbing (#45959)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-06-23 15:10:36 +03:00 |
|
  
|
6d8fff5698
|
[KV Connector][Offloading] Avoid blocking the engine to flush offloads on idle (#45595)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: Itay Etelis <Itay.etelis@gmail.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-06-17 11:35:07 +03:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Ronen Schafferandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
7e612a0f06
|
[KV Offloading] Implement reset_cache for TieringOffloadingManager (#44541)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-15 18:42:53 +00:00 |
|
Ronen Schaffer
|
2a2b5ca791
|
[KV Offload] Add on_schedule_end() hook to separate step lifecycle from event draining (#44206)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-02 13:42:52 +03:00 |
|
Ronen Schaffer
|
93da882e73
|
[kv_offload] Add @override decorators to subclass method implementations (#44177)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-02 08:07:47 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
a3ed5ab10c
|
[KV Offload] Add per-request offloading policy via on_new_request lifecycle hook (#43205)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-28 20:45:18 +00:00 |
|
Ronen Schaffer
|
4bfa0f2b14
|
[KV Offload] Rename SecondaryTierManager.get_finished() to get_finished_jobs() (#43870)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-28 16:00:18 +00:00 |
|
Ronen Schaffer
|
84747489de
|
Tier offload followup (#42529)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-18 19:41:58 +00:00 |
|
Ronen Schaffer
|
11f6b545d4
|
[kv_offload] Add multi-tier KV cache offloading framework (#40020)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-13 17:21:43 +03:00 |
|