 Sherif Walyandvllm-agent
|
ce8853d27a
|
[Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674)
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 02:11:12 +00:00 |
|
Sherif Waly
|
25385102fe
|
[Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU (#60071)
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
|
2026-10-06 14:25:08 +02:00 |
|
 Kam BasraandOpenAI Codex
|
0c16eee3f1
|
[Bugfix][KV Offload] Reset SimpleCPU eager-store placement state on MRV2 resume (#57816)
Signed-off-by: Kam Basra <kameldipbasra@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-05 02:00:07 +00:00 |
|
Zupeng Wang
|
18f8f96025
|
[Bugfix][KV Offload] Release pending CPU lookup pins on cache reset (#59862)
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
|
2026-10-04 04:30:17 +00:00 |
|
   
|
765872e7ed
|
[Core][BugFix] Tag prefix-cache extra keys by source (#51899)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Tai An <antai12232931@outlook.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-29 18:24:59 -07:00 |
|
mevince
|
cd1daf58a9
|
[Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConnector (#57251)
Signed-off-by: Vincent <vincexxchan@gmail.com>
|
2026-09-26 17:17:11 -07:00 |
|
 
|
927c87b347
|
[PCP][DCP] Support DCP target model with non-DCP Dspark (#56723)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-26 18:12:35 -04:00 |
|
 
|
b3a87b2d04
|
[Bugfix][KVConnector] Finalize saves on steps without a forward (#57775)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
|
2026-09-25 00:06:36 +00:00 |
|
 
|
f648eed23d
|
[Bugfix][KV Offload] Skip non-prefix-cacheable groups in SimpleCPUOffload (GLM-5.3-Flash kpool tail and QSA) (#56810)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-09-20 10:58:55 +00:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
Lucas Wilkinson
|
410f6da5c4
|
[Bugfix][KV Offload] Submit CPU stores on no-forward steps (#56621)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-09-13 05:56:14 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) mevinceandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c1b69aa0d4
|
[Bugfix][KV Connector] Only enforce disk block alignment for O_DIRECT (#55424)
Signed-off-by: Vincent <vincexxchan@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-11 13:12:59 -07:00 |
|
    
|
e77daef89e
|
[Model] Support DeepSeek-V4.1-Flash (#56214)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-11 02:11:19 -07:00 |
|
YukioZzz
|
07b7553465
|
[Feature][SimpleCPU] Load fine-grained hybrid prefix hits (#54736)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
|
2026-09-10 20:36:37 -05:00 |
|
 Andy LoandOpenAI Codex
|
5ebce2391e
|
[Bugfix][KV Offload] Respect prefix-cache bypass in SimpleCPUOffload (#54998)
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-08 13:14:19 +00:00 |
|
 He LiuandKimi
|
eb6b619ab2
|
[Bugfix][KV Offload] Register mixed page sizes in one cache group (#54756)
Signed-off-by: heliubj18 <chengfeng@ppio.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
|
2026-09-07 20:50:15 -07:00 |
|
mevince
|
4ae6228284
|
[Bugfix][KV Connector] Populate SimpleCPUOffload BlockStored metadata (#54325)
|
2026-09-03 15:58:07 +00:00 |
|
 Fabian JoswigandYifan Qiao
|
8052102c21
|
[Bugfix] Fix cross-batch buffer race corrupting DiskBackend loads (#51667)
Signed-off-by: Fabian Joswig <fjosw@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-09-02 08:18:41 +00:00 |
|
  
|
12b9573c98
|
[Bugfix][KV Offloading] Fix eager SimpleCPUOffload cache registration and final flush (#53532)
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-09-02 03:23:09 +00:00 |
|
 Ziming HuangandOpenAI Codex
|
481839ad9e
|
[Feature][Spec] Support disabling trailing prefix-cache block dropping (#53388)
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <48115868+ZeldaHuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-01 19:12:36 +08:00 |
|
 Summer YangandNick Hill
|
2aac565cae
|
[Core][KV Connector] Start async KV loads after the forward launch when no sync loads are scheduled (#53333)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-28 21:58:00 +00:00 |
|
 Jiangyun ZhuandOpenAI Codex
|
478ec3ea41
|
[Bugfix][KV Offload] Handle padded GPU cache storage (#54021)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-27 10:52:59 +00:00 |
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
 Hongxia YangandYifan Qiao
|
cd7b7c265a
|
[ROCm] Cpu offload for ROCm 7.13+ to align the hipMemcpyBatchAsync params and perf in 7.14x (#43018)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-21 09:07:03 -07:00 |
|
 Tyler Michael SmithandOpenAI Codex
|
017e9f4448
|
Promote prefix_cache_retention_interval to an argument and change the default to 0 (#52216)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-17 13:20:50 +00:00 |
|
 Guanyi ChenandClaude
|
58fcaa0baa
|
[Feat][Core] Add disk offloading support to SimpleCPUOffloadConnector (#49644)
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-07 15:37:47 +08:00 |
|
 Yifan QiaoandOpenAI Codex
|
9bc266d923
|
[Bugfix][KV Offload] Propagate EAGLE mode to SimpleCPU coordinator (#49071)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-20 06:39:11 +00:00 |
|
  
|
b6ff8a2f50
|
[Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-19 02:53:15 +00:00 |
|
Nick Hill
|
8ac8375270
|
[Core] Preserve Marconi caching with selective hybrid cache retention (#47782)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-13 21:24:20 +01:00 |
|
Chauncey
|
529af88842
|
[KV Offloading] Add free block iterator for CPU offload scheduling (#47849)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-07-09 06:59:54 +00:00 |
|
  
|
cd0de48d08
|
[Bugfix][V1] Free out-of-window blocks on the processed-token basis under async scheduling (#47728)
Signed-off-by: Saddss <28726669061@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Saddss <28726669061@qq.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-08 13:34:19 +01:00 |
|
Jeff (Junze) Ma
|
36fd7e8b86
|
[SimpleCPUOffloadConnector] Fix remaining global→block conversions under PCP/DCP (#46394)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
|
2026-06-24 23:05:24 -07:00 |
|
Saddss
|
82ede09a5a
|
[Bugfix][KVConnector] Fix SimpleCPUOffloadConnector GPU->CPU store race (#46278)
|
2026-06-22 13:08:47 -07:00 |
|
Nick Hill
|
7df3d7dada
|
[Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-20 20:02:24 -07:00 |
|
Jonathan Chen
|
c88d3d4775
|
[SimpleCPUOffloadConnector] PCP + DCP support (#39831)
Signed-off-by: Jonathan Chen <chenleejonathan@gmail.com>
|
2026-06-20 15:01:06 -07:00 |
|
 
|
cc22621b51
|
[KV Offload] Support packed HMA KV cache layout (#46205)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-06-20 21:19:40 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Jonathan Chenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d57888efa4
|
[SimpleCPUOffloadConnector]: Add support for reset_cache() (#39726)
Signed-off-by: Jonathan Chen <chenleejonathan@gmail.com>
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 19:47:12 -07:00 |
|
Yifan Qiao
|
7c37096620
|
[Core][Refactor]: thread scheduler_block_size into KVCacheManager and KVCacheCoordinator (#44165)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-02 01:14:44 -07:00 |
|
Yifan Qiao
|
81252d4e24
|
[Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (#42296)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-05-25 14:04:30 +08:00 |
|
 
|
fab07e4d0f
|
[Bugfix][KV Connector] Fix SimpleCPUOffloadScheduler TOCTOU between Phase A and Phase B (#42289)
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: gemini-code-assist <noreply@google.com>
|
2026-05-18 21:22:33 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
9ce74042d3
|
[Bugfix][SimpleCPUOffloadBackend] Dedup in-flight CPU offload stores across scheduler steps (#41289)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-13 01:53:32 -07:00 |
|
 Hongxia YangandTJian
|
20cac26b19
|
[ROCm] Enable SimpleCPUOffloadConnector on ROCm (#40549)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-05-06 20:52:02 -07:00 |
|
 
|
7b1bc0a3eb
|
[Bugfix] Cap SWA/chunked-local runtime admission to startup pool-sizing bound (#40946)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-27 04:33:13 +00:00 |
|
Yifan Qiao
|
91e4521f9f
|
[Feat][v1] Simple yet General CPU KV Cache Offloading (#37160)
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-03-31 17:58:37 -07:00 |
|