 Ilia Yastrebov
|
41848caa63
|
[NIXL] Use int32 array for indices to avoid intermediate conversion (#51952)
Signed-off-by: Ilia Yastrebov <iyastrebov@nvidia.com>
|
2026-09-02 16:30:16 +02:00 |
|
|
AlexHuangandOpenAI Codex
|
35faf957d6
|
[Bugfix][KV Offload] Ignore stale async lookup results (#54872)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-02 17:10:26 +03:00 |
|
|
Cheng RuiandCodex
|
2a4e3cc3db
|
[Bugfix][KV Offload] Ensure tracker progress for oversized offers (#54759)
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-02 14:09:29 +03:00 |
|
|
|
4c58a0c398
|
[Bugfix][KV Offload] Unlink /dev/shm region after all workers map it (barrier variant of #51317) (#52596)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-31 15:53:00 +03:00 |
|
|
Uttam KumarandOr Ozeri
|
e0d27040dd
|
[Bugfix][KV Offload][P2P] Preserve aborted loads until abort completion (#52571)
Signed-off-by: Uttam Kumar <uttam.kumar912@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-31 14:34:48 +03:00 |
|
|
liranschour
|
d8de4ae322
|
[Bugfix][KVOffload] P2P tier declares REQUEST_LEVEL on the producer leg (#52912)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
|
2026-08-31 10:07:20 +00:00 |
|
|
|
e6bfe03ad7
|
Count store offers, not lookups, for CPU offload store_threshold (#52227)
Signed-off-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Co-authored-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-27 20:21:06 -07:00 |
|
|
Almog Tavor
|
bc39ded3c4
|
[Bugfix][KV Offload] Defer request-level cascade of in-flight primary keys (#53329)
Signed-off-by: almogtavor <almogtavor@gmail.com>
|
2026-08-25 23:20:03 +00:00 |
|
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
|
Russell BryantandClaude Opus 4.8
|
ef47a897e2
|
[Core] Make prefix-cache NONE_HASH deterministic by default (#51875)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-18 23:14:39 +00:00 |
|
|
Ziqi Fan
|
f9a0f629bd
|
[KV Offload] Expose data-parallel topology to offloading backends (#51879)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2026-08-12 20:27:26 -07:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
f067737b2a
|
[Bugfix][TieredOffloading] : Return HIT_PENDING when KV promotion is triggered (#51840)
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-08-12 05:07:16 +03:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
1ab2801dde
|
[ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-11 14:19:16 -04:00 |
|
|
AlexHuang
|
0fec3d652b
|
[Bugfix][KV Offload] Centralize shared mmap cleanup in CPU worker (#51622)
Signed-off-by: Alex <jihui.huang@daocloud.io>
|
2026-08-11 13:22:15 +03:00 |
|
|
Itay EtelisandItay Etelis
|
81840a172f
|
[KV Connector] Canonical CPU layout for parallelism-agnostic KV offload (#48414)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-08-10 10:44:54 +03:00 |
|
|
Srinivas Krovvidiandsrinivas_oo7
|
a123159f7a
|
Add tiering offloading metrics (#48798)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-08-10 05:23:14 +03:00 |
|
|
Robbie J
|
1b0ce31f32
|
[KV Offload] Fix failed-load livelock by marking the lookup verdict as a miss (#49328)
Signed-off-by: Robbie J <RobbieJ@users.noreply.github.com>
|
2026-08-09 10:25:36 +03:00 |
|
|
|
ef2615c2e0
|
[Bugfix][KV Offload] Fall back when MADV_POPULATE_WRITE is unsupported (#51116)
Signed-off-by: Guy Girmonsky <guygir@gmail.com>
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: Guy Girmonsky <guygir@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-06 13:48:01 +03:00 |
|
|
|
9833aa53d5
|
[Bugfix] Fail fast with a clear error when CPU offload region exceeds available space (#50358)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-05 20:59:55 +03:00 |
|
|
Ronen SchafferandOr Ozeri
|
8543522ca7
|
[KV Offload] Support out-of-tree secondary tier managers via module_path (#51007)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-05 20:30:53 +03:00 |
|
|
MINJUN GIL
|
b92352ca2c
|
[Perf][KV Offload] Avoid quadratic ARC batch eviction (#50992)
Signed-off-by: MINJUN GIL <alswnsrlf12@naver.com>
|
2026-08-05 10:58:36 +03:00 |
|
|
|
41a7e7da0c
|
[KV Offload] Support partial secondary-tier load results (#50321)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-04 20:48:58 -07:00 |
|
|
Chang GuoandCursor Agent
|
541128bdeb
|
[KV Offload] Enable single-copy MLA layout for CPUOffloadingSpec (#50301)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
|
2026-07-31 06:20:47 +03:00 |
|
|
Chang GuoandCursor Agent
|
542a8fad6d
|
[KV Offload] Move CPUOffloadingSpec onto SharedOffloadRegion (#50094)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
|
2026-07-29 13:05:50 +03:00 |
|
|
Philip Pesic
|
dc1be79031
|
Add CachePolicyFactory for pluggable/external eviction policies (#49114)
Signed-off-by: Philip Pesic <philippesic06@gmail.com>
|
2026-07-29 07:34:55 +03:00 |
|
|
|
30217b0e80
|
[Bugfix][KV Offload][P2P] Scope serve state to fetch rounds (#49877)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-07-28 18:01:55 +03:00 |
|
|
MINJUN GILandOpenAI Codex
|
52c3c4a42f
|
[Bugfix][KV Offload][OBJ] Preserve job completion during cleanup (#49947)
Signed-off-by: MINJUN GIL <alswnsrlf12@naver.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-28 07:10:52 +03:00 |
|
|
|
a8f296083f
|
[KV Offload] Make compact secondary identity TP-independent (#49858)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: GPT-5.6 Sol <noreply@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-28 07:10:20 +03:00 |
|
|
Varun Sundar RabindranathandVarun Sundar Rabindranath
|
d18ed2304a
|
[KV-offload][FS] : Batch store/load_block in C (#49152)
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
|
2026-07-28 11:21:53 +08:00 |
|
|
Ronen Schaffer
|
77cba0259f
|
[KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-07-27 13:29:57 +03:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
394beb633b
|
[Bugfix][ROCm] Use batch DMA for CPU KV cache loads (#49843)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-27 02:18:01 -07:00 |
|
|
JasonandOr Ozeri
|
53397fbfac
|
[Bugfix][KV Offload][P2P] Fix EngineCore crash reconnecting to a reaped peer (#49823)
Signed-off-by: Jason Yao <wsyjh8@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-27 08:07:00 +03:00 |
|
|
|
da3a252fd1
|
[KVOffload][P2P] Generic P2P secondary tier: peer lookup and serving via ParentManager (#48021)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 11:45:47 +03:00 |
|
|
Chang GuoandOpenAI Codex
|
7eca0e1a64
|
[KV Offload] Deduplicate replicated MLA KV in the shared CPU region (#48906)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-26 08:22:30 +03:00 |
|
|
Jonguk CheongandOr Ozeri
|
7a29a3c54c
|
[Bugfix][KV Offload] Namespace persistent cache by model runner (#49440)
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 08:21:52 +03:00 |
|
|
Harry Mellor
|
89f6aa3a9e
|
[KV Offload][CI] Fall back to buffered I/O without O_DIRECT; fix flaky api-server test (#49734)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-24 13:56:57 -07:00 |
|
|
|
b6ff8a2f50
|
[Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-19 02:53:15 +00:00 |
|
|
|
426d48bfa1
|
[KV Offload] Add optional tier locality to FS/OBJ KV events (#48281)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-17 10:31:52 +03:00 |
|
|
Debasish MohantyandOr Ozeri
|
472d330c21
|
Add blocks_per_chunk configuration for KV offloading to support heterogeneous KV cache groups (#48878)
Signed-off-by: Debasish-87 <22btics06@suiit.ac.in>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-17 09:00:12 +03:00 |
|
|
Srinivas Krovvidiandsrinivas_oo7
|
ce65385618
|
[KV Offload] Split tiering_lookup_delay into sync/async histograms (#47679)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
|
2026-07-16 20:03:45 +03:00 |
|
|
|
a9531edfa6
|
[KV Offload] Define clean backend configuration boundary (#48150)
Signed-off-by: Change72 <changg@nvidia.com>
Signed-off-by: Chang Guo <cguo51@asu.edu>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-16 13:27:05 +03:00 |
|
|
liranschourandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
615834ee58
|
[KVOffload][P2P] Well-known default host/port env vars and per-DP-rank control port (#47636)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-15 15:22:56 +03:00 |
|
|
Chaojun Zhang
|
5811ed6a05
|
[Test][kv_offload] Fix flaky drain() helper in test_fs_tier.py (#48545)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-07-15 14:53:47 +03:00 |
|
|
|
cdaa40d2a8
|
[KV Offload] Split cpu_cache_usage_perc into write/read usage gauges (#47666)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-14 20:13:41 +03:00 |
|
|
liranschourandOr Ozeri
|
5c0c987c03
|
Make tiering offload region DP-replica aware (#47987)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-12 13:10:21 +03:00 |
|
|
|
2d814a0082
|
[kv_offload] Emit tier-owned BlockStored events from FS/OBJ secondary tiers (#47923)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-10 06:17:23 +03:00 |
|
|
Posedge_LinandOr Ozeri
|
0ca6eee743
|
[Core] Pass request context to CPU offload cache policy touch (#47744)
Signed-off-by: jacklin78911-collab <jacklin78911@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-08 05:56:25 +03:00 |
|
|
|
c46ced1ee3
|
[kv_offload] Establish tier-owned KV event handling (#46544)
Signed-off-by: Change72 <changg@nvidia.com>
Signed-off-by: Chang Guo <changg@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 16:55:20 +03:00 |
|
|
Pierangelo Di PilatoandOr Ozeri
|
65a7b46284
|
[KV-Offloading] Support workload identity for objectstore secondary tier (#47063)
Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 16:30:16 +03:00 |
|
|
|
e040899a00
|
[KV Offloading] Add basic offloading metrics (#45958)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Signed-off-by: Srinivas Krovvidi <194645829+Srinivasoo7@users.noreply.github.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 09:26:28 +03:00 |
|