 liranschourandKevin H. Luu
|
c5f8adc0b1
|
[KV Offload] Expose OffloadingManager.lock()/unlock() and service the tiering control plane off the scheduler thread (#58168)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-15 08:50:56 +03:00 |
|
 
|
775fefa804
|
[Metrics][KV Offload] Add vllm:kv_offload_config_info - generic info metrics for offloading the offloading connector (#56867)
Signed-off-by: Amir Friedman <Amir.Friedman1@ibm.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-10-14 12:34:37 +00:00 |
|
  
|
e37e51dd24
|
[Metrics] Expose cached prompt tokens by cache tier (#56318)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-06 14:52:05 -04:00 |
|
Karen Chung
|
986e217158
|
[Feature] Add first-class KV hints request envelope for programmatic KV management (#53423)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-09-21 09:56:11 -07:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
 MINJUN GILandOr Ozeri
|
f12fe10c5c
|
[Bugfix][KV Offload] Track cache recency once per request (#51787)
Signed-off-by: mindungil <alswnsrlf12@naver.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-16 10:47:26 +03:00 |
|
Ronen Schaffer
|
138d137b5b
|
[Refactor][kv_offload]: rename block→chunk (#52615)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-09-09 17:45:49 +03:00 |
|
Ilia Yastrebov
|
41848caa63
|
[NIXL] Use int32 array for indices to avoid intermediate conversion (#51952)
Signed-off-by: Ilia Yastrebov <iyastrebov@nvidia.com>
|
2026-09-02 16:30:16 +02:00 |
|
Moein Khazraee
|
9acbc5360a
|
[KV Offload] Preserve KV event metadata until final residency removal (#52068)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
|
2026-08-31 13:28:30 +03:00 |
|
 
|
f9d666f917
|
[KV Offload] Forward ownership in KV cache events (#52067)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-08-31 10:28:41 +03:00 |
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
  
|
2c4d348848
|
[KV Connector] Add per-layer canonical KV page mappings for parallelism-agnostic offload (#48408)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-07-31 14:42:54 +03:00 |
|
Philip Pesic
|
dc1be79031
|
Add CachePolicyFactory for pluggable/external eviction policies (#49114)
Signed-off-by: Philip Pesic <philippesic06@gmail.com>
|
2026-07-29 07:34:55 +03:00 |
|
Ronen Schaffer
|
77cba0259f
|
[KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-07-27 13:29:57 +03:00 |
|
 
|
da3a252fd1
|
[KVOffload][P2P] Generic P2P secondary tier: peer lookup and serving via ParentManager (#48021)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 11:45:47 +03:00 |
|
 Chang GuoandOpenAI Codex
|
7eca0e1a64
|
[KV Offload] Deduplicate replicated MLA KV in the shared CPU region (#48906)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-26 08:22:30 +03:00 |
|
 
|
426d48bfa1
|
[KV Offload] Add optional tier locality to FS/OBJ KV events (#48281)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-17 10:31:52 +03:00 |
|
   
|
a9531edfa6
|
[KV Offload] Define clean backend configuration boundary (#48150)
Signed-off-by: Change72 <changg@nvidia.com>
Signed-off-by: Chang Guo <cguo51@asu.edu>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-16 13:27:05 +03:00 |
|
 
|
c46ced1ee3
|
[kv_offload] Establish tier-owned KV event handling (#46544)
Signed-off-by: Change72 <changg@nvidia.com>
Signed-off-by: Chang Guo <changg@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 16:55:20 +03:00 |
|
 Ronen SchafferandOr Ozeri
|
0fc2512094
|
[KV Offload] Pass ScheduleEndContext to on_schedule_end hook (#46450)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-30 17:07:12 +03:00 |
|
 Ronen SchafferandOr Ozeri
|
bb61177e49
|
[KV Offloading] Replace bool|None lookup return with LookupResult enum (#46363)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-24 18:06:08 +03:00 |
|
 Rui YinandOr Ozeri
|
cf9fd6457e
|
Fix KV offload request-finished lifecycle contract (#46284)
Signed-off-by: test test <2260891073@qq.com>
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-24 15:42:40 +03:00 |
|
Martin Hickey
|
f237e16b41
|
[KV Offload] Replace OffloadingHandler with OffloadingWorker (#45053)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-06-24 14:44:24 +03:00 |
|
 Change72andClaude
|
a9f7b2d41c
|
[feature][kv_offload] Self-describing KV events for OffloadingConnector (#43468)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-22 07:27:46 +00:00 |
|
 Srinivas Krovvidiandsrinivas_oo7
|
c441ad1c07
|
[KV Offloading] Add labeled metrics support (#45957)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-06-21 18:04:01 +00:00 |
|
Ronen Schaffer
|
f428718ffe
|
[Fix][KV offload] Defer on_request_finished until in-flight transfers drain (#45823)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-18 07:05:46 +03:00 |
|
  
|
6d8fff5698
|
[KV Connector][Offloading] Avoid blocking the engine to flush offloads on idle (#45595)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: Itay Etelis <Itay.etelis@gmail.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-06-17 11:35:07 +03:00 |
|
 ![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)  
|
9dfc313bdc
|
Feature/offloading manager stats (#35669)
Signed-off-by: Sriusa4414@gmail.com
Signed-off-by: srinivas_oo7 <Sriusa4414@gmail.com>
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Signed-off-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-10 12:44:55 +00:00 |
|
Ronen Schaffer
|
2a2b5ca791
|
[KV Offload] Add on_schedule_end() hook to separate step lifecycle from event draining (#44206)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-02 13:42:52 +03:00 |
|
Ronen Schaffer
|
93da882e73
|
[kv_offload] Add @override decorators to subclass method implementations (#44177)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-02 08:07:47 +00:00 |
|
 Itay EtelisandItay Etelis
|
d63108fb18
|
[kv_offload] Skip decode-phase blocks in CPU offload (#43797)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-05-29 06:39:43 +03:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
a3ed5ab10c
|
[KV Offload] Add per-request offloading policy via on_new_request lifecycle hook (#43205)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-28 20:45:18 +00:00 |
|
Or Ozeri
|
357fddf614
|
[kv_offload]: Add DSv4 support (#43142)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
|
2026-05-24 11:10:12 +03:00 |
|
 ![gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)
|
5bd8c71e79
|
[kv_offload] Implement reset_cache() for the offloading connector (#41956)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
|
2026-05-14 16:00:10 +03:00 |
|
Ronen Schaffer
|
11f6b545d4
|
[kv_offload] Add multi-tier KV cache offloading framework (#40020)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-13 17:21:43 +03:00 |
|
Ronen Schaffer
|
79fd1bc7ed
|
[kv_offload] Add req_id to ReqContext for per-request tracking (#42507)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-13 11:11:10 +00:00 |
|
Ronen Schaffer
|
e175192d33
|
[KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() (#41366)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-10 15:09:25 +03:00 |
|
 Itay EtelisandItay Etelis
|
98661fe012
|
[Bugfix][KVConnector] Support DCP/PCP in OffloadingConnector (#41549)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-05-05 14:54:29 +03:00 |
|
Ronen Schaffer
|
415a879899
|
[KV Offload] Use Collection instead of Sequence/Iterable for OffloadingManager key parameters (#41361)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-05-01 05:18:38 +03:00 |
|
 Martin HickeyandOr Ozeri
|
c42981d034
|
[Refactor][kv_offload] KV Offloading maintainability improvements (#40538)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-04-30 05:55:31 +03:00 |
|