|
e77daef89e
|
[Model] Support DeepSeek-V4.1-Flash (#56214)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-11 02:11:19 -07:00 |
|
|
Kevin H. LuuandOpenAI Codex
|
1cf6555214
|
[Bugfix] Pin EPLB and MLA host-to-device transfer buffers (#56138)
Signed-off-by: khluu <khluu000@gmail.com>
Signed-off-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-11 01:57:47 -07:00 |
|
|
Simon Danielsson
|
ae48466cf3
|
[Perf][ROCm] Add AITER custom AG/RS (DP only) (#48247)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
|
2026-09-11 13:19:57 +08:00 |
|
|
|
ce08bb5b34
|
[Bugfix][Mooncake] Fix heterogeneous PP transfer completion (#56033)
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-11 10:57:19 +08:00 |
|
|
YukioZzz
|
07b7553465
|
[Feature][SimpleCPU] Load fine-grained hybrid prefix hits (#54736)
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
|
2026-09-10 20:36:37 -05:00 |
|
|
MattandClaude Opus 5
|
9163190dda
|
[ROCm][Bugfix][Perf] Tune multi-stream shared experts use; wvSplitKrc fixes (#56098)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-10 15:14:22 -07:00 |
|
|
|
7cdd9304ae
|
[KV Connector] Support symmetric DCP disagg for hybrid mamba models (#55531)
Signed-off-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster>
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-10 11:12:15 -07:00 |
|
|
|
8359e15aae
|
[ROCm][Feature] Support KV connectors with ROCM_AITER_UNIFIED_ATTN (#53695)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
Signed-off-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-09-10 12:48:42 -05:00 |
|
|
Lucas WilkinsonandNicolò Lucchesi
|
93911fcf68
|
[NIXL][PCP] Report replicated-PCP ranks > 0 as done sending instead of hiding them (#53903)
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
|
2026-09-10 12:58:05 +02:00 |
|
|
Ronen Schaffer
|
138d137b5b
|
[Refactor][kv_offload]: rename block→chunk (#52615)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-09-09 17:45:49 +03:00 |
|
|
|
9ffb8cea96
|
[CI] Add e2e test for scale-out EC connector flow (#54973)
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: zhouyou9505 <zhouyou9505@gmail.com>
Co-authored-by: Cursor Grok 4.6 <noreply@cursor.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-09 07:38:32 -07:00 |
|
|
Harry Huangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
3116c5d06b
|
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism (#54371)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-09 07:32:34 -07:00 |
|
|
Yicong WangandOpenAI Codex
|
d2906091bf
|
[Bugfix] Fix Mooncake heterogeneous TP with replicated GQA heads (#52516)
Signed-off-by: wangyicong <wangyicong@bytedance.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-09 14:18:32 +08:00 |
|
|
|
bf7eee9d7b
|
[Bugfix][EC Connector] Fix ECExampleConnector load device under TP>1 (#40416)
Signed-off-by: Hang Gong <kidogong@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-09 11:44:30 +08:00 |
|
|
Akine-Ko
|
41541636fd
|
[Refactor][EC Connector] Add backend extension points to ECCPUWorker (#54033)
Signed-off-by: Akine-Ko <2923601304@qq.com>
|
2026-09-08 20:13:47 -07:00 |
|
|
Tianyu GuoandOpenAI Codex
|
f1aeff6007
|
[Bugfix][EC Connector] Fail the request, not the engine, when a remote encoding cannot arrive (#55290)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-08 20:13:33 -07:00 |
|
|
|
0b066293f3
|
[Core][KV Connector] Resolve connector block tables for every scheduled request (#54853)
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-09 02:15:53 +00:00 |
|
|
Nick Hill
|
4aadb0f14d
|
[Core] Enhance cpu<->gpu sync checking to include paged async copies (#53491)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-08 17:26:21 -07:00 |
|
|
|
e0aaef85f3
|
[2/N][KV Connector][NIXL] Support per-region transfer geometry (#53780)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-08 16:01:12 -04:00 |
|
|
Wentao Ye
|
6b15bea080
|
[Refactor] Remove utils dead code (#53941)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-08 11:22:09 -04:00 |
|
|
Taoyu ZhuandShanshan Shen
|
ce6c241ddc
|
[ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path (#54787)
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-09-08 07:21:49 +00:00 |
|
|
ahmed xijiaat
|
81dabe2e5d
|
[Bugfix][KV Offload] Validate SWA coverage at unaligned cache-hit boundaries (#55712)
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com>
|
2026-09-08 10:15:48 +03:00 |
|
|
Kam BasraandClaude
|
4a806d08ee
|
[Bugfix] OffloadingConnector: stop zeroing offload hits under MTP/EAGLE spec decode (#52771)
Signed-off-by: Kam Basra <kameldipbasra@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-07 15:16:05 +03:00 |
|
|
AlexHuang
|
49eb2accf0
|
[KVConnector] Guard lmcache_mp_connector state transition with num_external_tokens (#47505)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: Alex-ai-future <Alex-ai-future@users.noreply.github.com>
|
2026-09-07 08:19:07 +00:00 |
|
|
Zhewen Liandzhewenl
|
34b1e9f7a6
|
[Bugfix][MooncakeStore] Fix finish-time save crash on hybrid models (#54643)
Signed-off-by: zhewenl <zhewenl@users.noreply.github.com>
Co-authored-by: zhewenl <zhewenl@users.noreply.github.com>
|
2026-09-07 07:18:30 +00:00 |
|
|
|
6fbb00b188
|
[EPD] Add ECMooncakeConnector for encoder cache over Mooncake TransferEngine (#41567)
Signed-off-by: Teng Ma <sima.mt@alibaba-inc.com>
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-07 06:21:43 +00:00 |
|
|
Almog TavorandOr Ozeri
|
dc02934a07
|
[Bugfix][KV Offload] Stop offloading the final sampled token's KV slot (#54288)
Signed-off-by: almogtavor <almogtavor@gmail.com>
Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
|
2026-09-06 15:22:19 +03:00 |
|
|
|
569adb5a97
|
[Bugfix][KV Offload] Fix SWA store reachability during chunked prefill (#54362)
Signed-off-by: Will Hampson <whamp@ggl.slmail.me>
Co-authored-by: Will Hampson <whamp@ggl.slmail.me>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-06 13:50:39 +03:00 |
|
|
Nick Hill
|
8277c42e4c
|
[Perf] Ensure async h2d copies are pinned in more places (#55202)
|
2026-09-04 17:17:01 -07:00 |
|
|
|
eb74fbb3e7
|
[Bugfix][NIXL] Don't assert when a failed transfer is cleaned up twice (#54518)
Signed-off-by: Yizheng Jiao <jyizheng@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-04 23:20:32 +00:00 |
|
|
Ilya MarkovandBugen Zhao
|
ae2d1ca96f
|
[Rust Frontend] Record Mooncake/NIXL KV-connector metrics (#52755)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-04 06:21:54 +00:00 |
|
|
Jeff (Junze) MaandOpenAI Codex
|
e35298628f
|
[Bugfix][KV Connector] Fix DecodeBenchConnector prefix block selection (#54878)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-04 04:05:58 +00:00 |
|
|
omerpaz95andClaude Opus 5
|
8a0a7ee40a
|
[EC Connector] P2P NIXL + CPU EC Connector (#47941)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-04 11:23:56 +08:00 |
|
|
bnellnmandTyler Michael Smith
|
25268f0c9c
|
[KVConnector] Add retention interval to OffloadingConnector (#51886)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-09-04 02:42:20 +00:00 |
|
|
|
d9e2b5238d
|
[Core][KV Events] Echo session_id on GPU BlockStored events (#51381)
Signed-off-by: liuguangli <liuguangli35@gmail.com>
Signed-off-by: jinx-machinepulse <2679622408@qq.com>
Co-authored-by: liuguangli <liuguangli35@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-04 02:22:50 +00:00 |
|
|
Jiangyun Zhu
|
98ed0856f3
|
[Model] add GLM-5.3-Flash support (#53906)
|
2026-09-03 09:40:35 -07:00 |
|
|
Yifan Jiang
|
da8ec28268
|
[Bugfix][KV Offload] Do not let a recurrent group's unhashed block truncate the load boundary (#52807)
Signed-off-by: Yifan Jiang <19356972+yifjiang@users.noreply.github.com>
|
2026-09-03 19:08:46 +03:00 |
|
|
mevince
|
4ae6228284
|
[Bugfix][KV Connector] Populate SimpleCPUOffload BlockStored metadata (#54325)
|
2026-09-03 15:58:07 +00:00 |
|
|
Jeff (Junze) MaandOpenAI Codex
|
31e9c13685
|
[Bugfix][KV Connector] Safely fill circular buffers in DecodeBench (#54879)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-03 15:16:22 +00:00 |
|
|
|
092334c828
|
[Bugfix] Wait for offload keys before storing chunks (#52923)
Signed-off-by: chuanzhisongshu <49055103+982945902@users.noreply.github.com>
Co-authored-by: lishuo121 <lishuo121@jd.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-03 11:07:35 +03:00 |
|
|
Ilia Yastrebov
|
41848caa63
|
[NIXL] Use int32 array for indices to avoid intermediate conversion (#51952)
Signed-off-by: Ilia Yastrebov <iyastrebov@nvidia.com>
|
2026-09-02 16:30:16 +02:00 |
|
|
drakoshaandClaude
|
ba6c60e98f
|
[Bugfix][KV Offload] Scale UniformTypeKVCacheSpecs groups by DCP (#50883)
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-02 17:11:48 +03:00 |
|
|
Nicolò Lucchesiandcjackal
|
f81eb41934
|
[Bugfix] adjust_dcp_kv_cache_interleave_size for NixlConnector only (#54803)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
|
2026-09-02 12:31:14 +02:00 |
|
|
|
df09c76735
|
[KV Connector][Offloading] Look through UniformTypeKVCacheSpecs in the canonical portability gate (#51690)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-02 10:43:26 +03:00 |
|
|
|
12b9573c98
|
[Bugfix][KV Offloading] Fix eager SimpleCPUOffload cache registration and final flush (#53532)
Signed-off-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Co-authored-by: maithilijoshi20 <maithilijoshi2003@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-09-02 03:23:09 +00:00 |
|
|
Jeff (Junze) MaandOpenAI Codex
|
18c53727ce
|
[Bugfix][KV Connector] Fix DecodeBench DCP block selection (#54679)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-01 20:31:30 +00:00 |
|
|
Zhewen Liandzhewenl
|
5cc32fbdfa
|
[Bugfix][KV Connector] Fix Mooncake physical-block transfer length (#54272)
Signed-off-by: zhewenl <zhewenl@users.noreply.github.com>
Co-authored-by: zhewenl <zhewenl@users.noreply.github.com>
|
2026-09-01 13:23:46 -07:00 |
|
|
|
7cb9a88f50
|
[Bugfix][Mooncake] Offload producer partial tails on request finish (#52832)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-01 13:23:35 -07:00 |
|
|
周峥andGuanyi Chen
|
c866ba9d11
|
[KV Connector] Support heterogeneous TP sharing in Mooncake Store Connector (#53129)
Signed-off-by: Guanyi Chen <939416532@qq.com>
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Zheng Zhou <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: zane <zhouzh93@mail2.sysu.edu.cn>
Co-authored-by: Guanyi Chen <939416532@qq.com>
|
2026-09-01 10:18:06 -07:00 |
|
|
Ziming HuangandOpenAI Codex
|
481839ad9e
|
[Feature][Spec] Support disabling trailing prefix-cache block dropping (#53388)
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <48115868+ZeldaHuang@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-01 19:12:36 +08:00 |
|