   
|
e10c10a644
|
[Core] Add PROMOTION_LATENCY histogram metric for tiering offload (#53910)
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-10-15 13:47:41 +03:00 |
|
 
|
775fefa804
|
[Metrics][KV Offload] Add vllm:kv_offload_config_info - generic info metrics for offloading the offloading connector (#56867)
Signed-off-by: Amir Friedman <Amir.Friedman1@ibm.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-10-14 12:34:37 +00:00 |
|
  
|
e37e51dd24
|
[Metrics] Expose cached prompt tokens by cache tier (#56318)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-06 14:52:05 -04:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
 MINJUN GILandOr Ozeri
|
f12fe10c5c
|
[Bugfix][KV Offload] Track cache recency once per request (#51787)
Signed-off-by: mindungil <alswnsrlf12@naver.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-16 10:47:26 +03:00 |
|
Ronen Schaffer
|
138d137b5b
|
[Refactor][kv_offload]: rename block→chunk (#52615)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-09-09 17:45:49 +03:00 |
|
liranschour
|
d8de4ae322
|
[Bugfix][KVOffload] P2P tier declares REQUEST_LEVEL on the producer leg (#52912)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
|
2026-08-31 10:07:20 +00:00 |
|
Almog Tavor
|
bc39ded3c4
|
[Bugfix][KV Offload] Defer request-level cascade of in-flight primary keys (#53329)
Signed-off-by: almogtavor <almogtavor@gmail.com>
|
2026-08-25 23:20:03 +00:00 |
|
 Varun Sundar RabindranathandVarun Sundar Rabindranath
|
f067737b2a
|
[Bugfix][TieredOffloading] : Return HIT_PENDING when KV promotion is triggered (#51840)
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-08-12 05:07:16 +03:00 |
|
 Srinivas Krovvidiandsrinivas_oo7
|
a123159f7a
|
Add tiering offloading metrics (#48798)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-08-10 05:23:14 +03:00 |
|
Robbie J
|
1b0ce31f32
|
[KV Offload] Fix failed-load livelock by marking the lookup verdict as a miss (#49328)
Signed-off-by: Robbie J <RobbieJ@users.noreply.github.com>
|
2026-08-09 10:25:36 +03:00 |
|
 
|
41a7e7da0c
|
[KV Offload] Support partial secondary-tier load results (#50321)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-04 20:48:58 -07:00 |
|
Ronen Schaffer
|
77cba0259f
|
[KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-07-27 13:29:57 +03:00 |
|
 Srinivas Krovvidiandsrinivas_oo7
|
ce65385618
|
[KV Offload] Split tiering_lookup_delay into sync/async histograms (#47679)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-07-16 20:03:45 +03:00 |
|
 
|
c46ced1ee3
|
[kv_offload] Establish tier-owned KV event handling (#46544)
Signed-off-by: Change72 <changg@nvidia.com>
Signed-off-by: Chang Guo <changg@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 16:55:20 +03:00 |
|
 Ronen SchafferandOr Ozeri
|
0fc2512094
|
[KV Offload] Pass ScheduleEndContext to on_schedule_end hook (#46450)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-30 17:07:12 +03:00 |
|
 Ronen SchafferandOr Ozeri
|
bb61177e49
|
[KV Offloading] Replace bool|None lookup return with LookupResult enum (#46363)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-24 18:06:08 +03:00 |
|
 Rui YinandOr Ozeri
|
cf9fd6457e
|
Fix KV offload request-finished lifecycle contract (#46284)
Signed-off-by: test test <2260891073@qq.com>
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-24 15:42:40 +03:00 |
|
 Srinivas Krovvidiandsrinivas_oo7
|
091bc1026e
|
[KV Offloading] Add tiering metric plumbing (#45959)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-06-23 15:10:36 +03:00 |
|
 Varun Sundar RabindranathandVarun Sundar Rabindranath
|
68567ef2df
|
[CPUOffloadingManager] Maintain evictable list in LRUCachePolicy (#46216)
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-06-22 06:54:44 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Ronen Schafferandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
7e612a0f06
|
[KV Offloading] Implement reset_cache for TieringOffloadingManager (#44541)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-15 18:42:53 +00:00 |
|
Rotem Shavitt
|
3f0a91bb96
|
Nit Changes in Tiered KV Offload (#44293)
Signed-off-by: Rotem Shavitt <rshavitt@gmail.com>
|
2026-06-02 21:53:21 -07:00 |
|