 Sherif Walyandvllm-agent
|
ce8853d27a
|
[Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts (#61674)
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
Co-authored-by: vllm-agent <claw@inferact.ai>
|
2026-10-17 02:11:12 +00:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
2cc2e68d1c
|
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-13 00:58:59 -07:00 |
|
 
|
df6c7ad031
|
[Core][Refactor] Move KV cache planning logic to kv_cache_planning.py (#61183)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-13 03:05:39 +00:00 |
|
 Woosuk KwonandClaude Opus 5.5
|
e1a5086256
|
[Attention] Remove cascade attention (#61126)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 10:41:07 -07:00 |
|
     
|
02e55d2af3
|
[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-09 15:48:37 -04:00 |
|
  
|
e37e51dd24
|
[Metrics] Expose cached prompt tokens by cache tier (#56318)
Signed-off-by: Cam Quilici <cjquilici@gmail.com>
Signed-off-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Cam Quilici <cameron@semianalysis.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-10-06 14:52:05 -04:00 |
|
Sherif Waly
|
25385102fe
|
[Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU (#60071)
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai>
|
2026-10-06 14:25:08 +02:00 |
|
 Kam BasraandOpenAI Codex
|
0c16eee3f1
|
[Bugfix][KV Offload] Reset SimpleCPU eager-store placement state on MRV2 resume (#57816)
Signed-off-by: Kam Basra <kameldipbasra@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-05 02:00:07 +00:00 |
|
Zupeng Wang
|
18f8f96025
|
[Bugfix][KV Offload] Release pending CPU lookup pins on cache reset (#59862)
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
|
2026-10-04 04:30:17 +00:00 |
|
 
|
cb6458c5ce
|
[Core] Remove deprecated mamba_cache_mode "all" (#58997)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
|
2026-10-01 02:59:37 -07:00 |
|
 Harry MellorandClaude Opus 5.5
|
fc2c801ad9
|
Disable mypy arg-type and assignment checks in tests (#59428)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-30 15:18:50 +00:00 |
|
 
|
5dd693122c
|
[MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-29 21:27:45 +01:00 |
|
 
|
927c87b347
|
[PCP][DCP] Support DCP target model with non-DCP Dspark (#56723)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-26 18:12:35 -04:00 |
|
 
|
f648eed23d
|
[Bugfix][KV Offload] Skip non-prefix-cacheable groups in SimpleCPUOffload (GLM-5.3-Flash kpool tail and QSA) (#56810)
Signed-off-by: Jared Wen <jaredwen@inferact.ai>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-09-20 10:58:55 +00:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
Lucas Wilkinson
|
410f6da5c4
|
[Bugfix][KV Offload] Submit CPU stores on no-forward steps (#56621)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-09-13 05:56:14 +00:00 |
|
      
|
d43bb2f37f
|
[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Matthew Bonanni <matthew@neuralmagic.com>
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-12 00:36:47 -04:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) mevinceandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c1b69aa0d4
|
[Bugfix][KV Connector] Only enforce disk block alignment for O_DIRECT (#55424)
Signed-off-by: Vincent <vincexxchan@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-11 13:12:59 -07:00 |
|
    
|
e77daef89e
|
[Model] Support DeepSeek-V4.1-Flash (#56214)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-11 02:11:19 -07:00 |
|
YukioZzz
|
07b7553465
|
[Feature][SimpleCPU] Load fine-grained hybrid prefix hits (#54736)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
|
2026-09-10 20:36:37 -05:00 |
|
 Andy LoandOpenAI Codex
|
5ebce2391e
|
[Bugfix][KV Offload] Respect prefix-cache bypass in SimpleCPUOffload (#54998)
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-08 13:14:19 +00:00 |
|
 He LiuandKimi
|
eb6b619ab2
|
[Bugfix][KV Offload] Register mixed page sizes in one cache group (#54756)
Signed-off-by: heliubj18 <chengfeng@ppio.com>
Co-authored-by: Kimi <noreply@moonshot.cn>
|
2026-09-07 20:50:15 -07:00 |
|
mevince
|
4ae6228284
|
[Bugfix][KV Connector] Populate SimpleCPUOffload BlockStored metadata (#54325)
|
2026-09-03 15:58:07 +00:00 |
|
  
|
12b9573c98
|
[Bugfix][KV Offloading] Fix eager SimpleCPUOffload cache registration and final flush (#53532)
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-09-02 03:23:09 +00:00 |
|
 Summer YangandNick Hill
|
2aac565cae
|
[Core][KV Connector] Start async KV loads after the forward launch when no sync loads are scheduled (#53333)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-28 21:58:00 +00:00 |
|
 Jiangyun ZhuandOpenAI Codex
|
478ec3ea41
|
[Bugfix][KV Offload] Handle padded GPU cache storage (#54021)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-27 10:52:59 +00:00 |
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
 Hongxia YangandYifan Qiao
|
cd7b7c265a
|
[ROCm] Cpu offload for ROCm 7.13+ to align the hipMemcpyBatchAsync params and perf in 7.14x (#43018)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-21 09:07:03 -07:00 |
|
 Nick HillandClaude
|
74295e3bd4
|
[CI][Bugfix] Fix flaky test_store_orders_after_compute_write (#50926)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-04 00:54:49 +00:00 |
|
  
|
b6ff8a2f50
|
[Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-19 02:53:15 +00:00 |
|
Jeff (Junze) Ma
|
36fd7e8b86
|
[SimpleCPUOffloadConnector] Fix remaining global→block conversions under PCP/DCP (#46394)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
|
2026-06-24 23:05:24 -07:00 |
|
Saddss
|
82ede09a5a
|
[Bugfix][KVConnector] Fix SimpleCPUOffloadConnector GPU->CPU store race (#46278)
|
2026-06-22 13:08:47 -07:00 |
|
Jonathan Chen
|
c88d3d4775
|
[SimpleCPUOffloadConnector] PCP + DCP support (#39831)
Signed-off-by: Jonathan Chen <chenleejonathan@gmail.com>
|
2026-06-20 15:01:06 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Jonathan Chenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d57888efa4
|
[SimpleCPUOffloadConnector]: Add support for reset_cache() (#39726)
Signed-off-by: Jonathan Chen <chenleejonathan@gmail.com>
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 19:47:12 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
0c6631f02a
|
[KVCache] Support Pluggable KVCacheSpec (#37505)
Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-03 09:05:16 -07:00 |
|
Yifan Qiao
|
81252d4e24
|
[Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (#42296)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-05-25 14:04:30 +08:00 |
|
 
|
fab07e4d0f
|
[Bugfix][KV Connector] Fix SimpleCPUOffloadScheduler TOCTOU between Phase A and Phase B (#42289)
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: gemini-code-assist <noreply@google.com>
|
2026-05-18 21:22:33 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
9ce74042d3
|
[Bugfix][SimpleCPUOffloadBackend] Dedup in-flight CPU offload stores across scheduler steps (#41289)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-13 01:53:32 -07:00 |
|
 Hongxia YangandTJian
|
20cac26b19
|
[ROCm] Enable SimpleCPUOffloadConnector on ROCm (#40549)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
|
2026-05-06 20:52:02 -07:00 |
|
Yifan Qiao
|
91e4521f9f
|
[Feat][v1] Simple yet General CPU KV Cache Offloading (#37160)
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-03-31 17:58:37 -07:00 |
|