 Ziming Huang
|
5b1e7a812b
|
[BugFix][Mooncake] Fix Mooncake saves from sparse Mamba block tables (#51362)
Signed-off-by: ZeldaHuang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
|
2026-08-20 03:05:43 -07:00 |
|
|
Guanyi Chenandz-zanez
|
bf2866f8bf
|
[KV Connector] Add decode offloading to Mooncake Store consumers (#52466)
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
|
2026-08-19 17:40:35 -07:00 |
|
|
Tyler Michael SmithandOpenAI Codex
|
017e9f4448
|
Promote prefix_cache_retention_interval to an argument and change the default to 0 (#52216)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-17 13:20:50 +00:00 |
|
|
Guanyi Chen
|
dc9ae4b8ac
|
[Bugfix][Mooncake] Reference GPU blocks for in-flight store jobs and key the store ledger by store_job_id (#52372)
Signed-off-by: Guanyi Chen <939416532@qq.com>
|
2026-08-16 16:09:46 -07:00 |
|
|
Nick HillandClaude Opus 4.8
|
615d4cfade
|
[Core] Check for GPU<->CPU syncs during CI (#43107)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 17:42:05 -07:00 |
|
|
Tzu-Ling KanandClaude Opus 4.8
|
653cc6faca
|
[Bugfix][NIXL] Include transfer mode (push/pull) in the compatibility hash (#50620)
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 12:11:22 +08:00 |
|
|
Ziqi Fan
|
15227b934a
|
[Bugfix][KV Offload] Emit self-describing CPU events at KV-group block granularity (#51614)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2026-08-12 20:28:15 -07:00 |
|
|
Ziqi Fan
|
f9a0f629bd
|
[KV Offload] Expose data-parallel topology to offloading backends (#51879)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2026-08-12 20:27:26 -07:00 |
|
|
liuzhenweiandKunshang Ji
|
d648236199
|
[XPU][CI] Fix ExampleConnector KV cache device selection (#51806)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-08-11 23:40:29 +08:00 |
|
|
Itay EtelisandItay Etelis
|
0f2ea973be
|
[KV Connector][Offloading] Keep per-layer KV registration when canonical_layout is requested (#51688)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-08-11 06:03:59 +03:00 |
|
|
Itay EtelisandItay Etelis
|
81840a172f
|
[KV Connector] Canonical CPU layout for parallelism-agnostic KV offload (#48414)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-08-10 10:44:54 +03:00 |
|
|
Yifan QiaoandOpenAI Codex
|
d6941300fc
|
[BugFix] Scope divergent hybrid cache hits to capable connectors (#50344)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-09 15:38:13 -07:00 |
|
|
ChaunceyandOr Ozeri
|
c423998642
|
[KV Offload] Emit self-describing events for partial recurrent blocks (#51243)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-09 11:28:44 +03:00 |
|
|
|
eb24bc38cf
|
[Bugfix][KV Offload] Handle chunked local attention in offloading scheduler (#51161)
Signed-off-by: almogtavor <almogtavor@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-09 10:25:57 +03:00 |
|
|
Nick Hill
|
9b0afeb4f6
|
[Perf] Avoid some more unnecessary GPU<->CPU syncs (#51458)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-08 17:13:26 -07:00 |
|
|
|
47228db84c
|
[Bugfix][KV-transfer] MoRIIO: per-layer READ-completion barrier in wait_for_layer_load (#48534)
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
Signed-off-by: Redwan Khan <redwan.khan@mangoboost.io>
Signed-off-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io>
Signed-off-by: limeward <32970461+edwinlim0919@users.noreply.github.com>
Signed-off-by: QinPR <1905873179@qq.com>
Co-authored-by: Jaeyoun Kim <jaeyoun.kim@mangoboost.io>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Redwan Khan <redwan.khan@mangoboost.io>
Co-authored-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io>
Co-authored-by: rkhan055 <redwankhan055@gmail.com>
Co-authored-by: Peiran Qin <66068739+QinPR@users.noreply.github.com>
Co-authored-by: QinPR <1905873179@qq.com>
|
2026-08-07 14:23:33 +00:00 |
|
|
Nicolò Lucchesi
|
d4ecb75ba2
|
[PD][NixlPush][Bugfix] Fix prefix caching (#48758)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-07 16:21:17 +02:00 |
|
|
Nicolò Lucchesi
|
5ec47f3e48
|
[PD][PushConnector] Record last activity of remotes to allow clean up of stale ones (#50234)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-07 09:43:29 +00:00 |
|
|
Guanyi ChenandClaude
|
58fcaa0baa
|
[Feat][Core] Add disk offloading support to SimpleCPUOffloadConnector (#49644)
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-07 15:37:47 +08:00 |
|
|
Qianxu Wangandqianxu
|
c810e5ee99
|
[Bugfix] Fix Mamba all-mode CPU offload boundary alignment (#51100)
Signed-off-by: qianxu <admin@picasso-lab.com>
Co-authored-by: qianxu <admin@picasso-lab.com>
|
2026-08-07 07:37:48 +03:00 |
|
|
LZWandRoger Wang
|
d36f24b66a
|
[KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector (#48069)
Signed-off-by: lzw <2994587066@qq.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-08-05 17:31:18 -07:00 |
|
|
ChaunceyandOr Ozeri
|
62c5e21621
|
[KV Offloading] Support partial-tail prefix reuse with fine-grained prefix matching (#50507)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-05 22:31:50 +08:00 |
|
|
|
fd859d2e06
|
[KV Connector][Mooncake] Add store group semantics (#44956)
Signed-off-by: Schatten <czhengt@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-05 08:25:38 +00:00 |
|
|
Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ffee32460d
|
[BugFix][Mooncake] Use global data_parallel_index for the DP engine index (#48061)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-04 17:10:51 -07:00 |
|
|
|
f42761204f
|
llmd+vllm+mori-ep(inter node wide-ep)+mori-io(write) for 2p2d with dp=ep=16 tp=1 (#45043)
Signed-off-by: Shiksha Patel <shikpate@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-04 00:13:15 +00:00 |
|
|
|
952694e384
|
[Bugfix] Validate NIXL speculative config compatibility (#49230)
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: GPT-5.6 Sol <noreply@openai.com>
Co-authored-by: Cursor Grok 4.5 <noreply@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-04 02:51:04 +08:00 |
|
|
Yifan Qiaoandxijiade.aihemaiti
|
5c4fe4b17e
|
[Bugfix][KV Connector] Propagate EAGLE state across merged Mooncake store groups (#49069)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xijiade.aihemaiti <3146335281@qq.com>
|
2026-08-02 19:32:26 -07:00 |
|
|
Jeff (Junze) MaandYifan Qiao
|
3986b967b2
|
(feat): optionally disable lookup on PD decode (#50498)
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-01 09:41:43 +00:00 |
|
|
|
2c4d348848
|
[KV Connector] Add per-layer canonical KV page mappings for parallelism-agnostic offload (#48408)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-07-31 14:42:54 +03:00 |
|
|
bnellnmandHarry Mellor
|
6e311c6e20
|
[MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-31 11:28:18 +01:00 |
|
|
|
4e582c5b54
|
[PD][Bugfix] Rebase KV lease deadlines onto worker clock (#50326)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-07-30 01:25:42 +00:00 |
|
|
Nick HillandClaude Opus 5
|
437e0b7f8e
|
[BugFix] Fix P/D preemption race condition (#50297)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-07-30 01:11:18 +00:00 |
|
|
Nick Hill
|
82553692b8
|
[KV Connector] Fix NIXL mamba state pairing for multi-slot block tables (#50153)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-29 09:45:53 -07:00 |
|
|
Nick HillandJared Wen
|
f37f03db4a
|
[KV Connector] Support NIXL P/D for hybrid MLA+SSM models (#49762)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Jared Wen <w13431838023@gmail.com>
|
2026-07-29 11:50:30 +08:00 |
|
|
Jonguk CheongandOpenAI Codex
|
1e81853afc
|
[Bugfix][KV Offload] Keep Mamba block span unscaled under DCP (#49964)
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-28 17:41:55 +03:00 |
|
|
Nick HillandClaude Fable 5
|
601fa9a74e
|
[KV Connector] Support NIXL heterogeneous P/D block sizes for hybrid models (#49612)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-28 06:45:02 -07:00 |
|
|
Nicolò Lucchesi
|
98e91a9600
|
[PD][NixlPush] Skip extra add_remote_agent step in D->P handshake (#49345)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-28 14:57:12 +02:00 |
|
|
|
1206891822
|
[ROCm][KVConnector][MoRI-IO] Fix WRITE-mode remote-TP rank collapse (#46332 follow-up) (#47764)
Signed-off-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Co-authored-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
|
2026-07-27 20:26:24 -05:00 |
|
|
Rishi PuriandClaude Opus 4.8
|
fd10e8946d
|
[Test] Regression test for hybrid-Mamba eagle cache-peek in Mooncake connector (#43559) (#48361)
Signed-off-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-27 19:03:58 +00:00 |
|
|
Nicolò Lucchesi
|
2b465b2c42
|
[Misc][PD] Nixl cleanup get_backend_aware_kv_block_len and virtually_split_kv_in_blocks (#49988)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-27 18:51:37 +02:00 |
|
|
Ronen Schaffer
|
77cba0259f
|
[KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-07-27 13:29:57 +03:00 |
|
|
Dao007foreverandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d742856610
|
[3/N][Core][KV Connector] Support reliable partial-tail KV offload for sub-block prompts (#49502)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-26 23:22:17 -07:00 |
|
|
limeward
|
ffc4f08c8e
|
[Core][KV-transfer] MoRIIO: heterogeneous TP<->DP prefill/decode read routing (#46116)
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
|
2026-07-27 02:10:00 +00:00 |
|
|
|
b68d7ef262
|
[Bugfix][KV Offload] Namespace auto cache dtype by effective dtype (#49438)
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 20:59:09 +03:00 |
|
|
Daniel SocekandKunshang Ji
|
7154856f3d
|
[Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive (#47791)
Signed-off-by: Daniel Socek <daniel.socek@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-26 22:22:42 +08:00 |
|
|
|
3f1d40960f
|
[KV Offload] Fix num_tokens_after_batch for different termination types (#49285)
Signed-off-by: Alex <jihuihuang@example.com>
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: Alex <jihuihuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 16:22:58 +03:00 |
|
|
|
5559679229
|
[Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks (#49052)
Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: jasl <jasl9187@hotmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 14:53:32 +03:00 |
|
|
Chang GuoandOpenAI Codex
|
7eca0e1a64
|
[KV Offload] Deduplicate replicated MLA KV in the shared CPU region (#48906)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-26 08:22:30 +03:00 |
|
|
Rui YinandOr Ozeri
|
d30b1ecd1b
|
[Bugfix][KV Offloading] Defer request finalization until final store (#49671)
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-25 20:58:09 +00:00 |
|
|
Yifan QiaoandClaude
|
70052fb924
|
[Bugfix][KV Connector][Mooncake] Keep TP-sharded Mamba state out of the KV-head dedup (#49499)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-24 21:02:22 -07:00 |
|