|
4e582c5b54
|
[PD][Bugfix] Rebase KV lease deadlines onto worker clock (#50326)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-07-30 01:25:42 +00:00 |
|
|
Nick HillandClaude Opus 5
|
437e0b7f8e
|
[BugFix] Fix P/D preemption race condition (#50297)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-07-30 01:11:18 +00:00 |
|
|
Nick Hill
|
82553692b8
|
[KV Connector] Fix NIXL mamba state pairing for multi-slot block tables (#50153)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-29 09:45:53 -07:00 |
|
|
Nick HillandJared Wen
|
f37f03db4a
|
[KV Connector] Support NIXL P/D for hybrid MLA+SSM models (#49762)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Jared Wen <w13431838023@gmail.com>
|
2026-07-29 11:50:30 +08:00 |
|
|
Jonguk CheongandOpenAI Codex
|
1e81853afc
|
[Bugfix][KV Offload] Keep Mamba block span unscaled under DCP (#49964)
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-28 17:41:55 +03:00 |
|
|
Nick HillandClaude Fable 5
|
601fa9a74e
|
[KV Connector] Support NIXL heterogeneous P/D block sizes for hybrid models (#49612)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-28 06:45:02 -07:00 |
|
|
Nicolò Lucchesi
|
98e91a9600
|
[PD][NixlPush] Skip extra add_remote_agent step in D->P handshake (#49345)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-28 14:57:12 +02:00 |
|
|
|
1206891822
|
[ROCm][KVConnector][MoRI-IO] Fix WRITE-mode remote-TP rank collapse (#46332 follow-up) (#47764)
Signed-off-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Co-authored-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
|
2026-07-27 20:26:24 -05:00 |
|
|
Rishi PuriandClaude Opus 4.8
|
fd10e8946d
|
[Test] Regression test for hybrid-Mamba eagle cache-peek in Mooncake connector (#43559) (#48361)
Signed-off-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-27 19:03:58 +00:00 |
|
|
Nicolò Lucchesi
|
2b465b2c42
|
[Misc][PD] Nixl cleanup get_backend_aware_kv_block_len and virtually_split_kv_in_blocks (#49988)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-27 18:51:37 +02:00 |
|
|
Ronen Schaffer
|
77cba0259f
|
[KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-07-27 13:29:57 +03:00 |
|
|
Dao007foreverandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d742856610
|
[3/N][Core][KV Connector] Support reliable partial-tail KV offload for sub-block prompts (#49502)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-26 23:22:17 -07:00 |
|
|
limeward
|
ffc4f08c8e
|
[Core][KV-transfer] MoRIIO: heterogeneous TP<->DP prefill/decode read routing (#46116)
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
|
2026-07-27 02:10:00 +00:00 |
|
|
|
b68d7ef262
|
[Bugfix][KV Offload] Namespace auto cache dtype by effective dtype (#49438)
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 20:59:09 +03:00 |
|
|
Daniel SocekandKunshang Ji
|
7154856f3d
|
[Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive (#47791)
Signed-off-by: Daniel Socek <daniel.socek@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-26 22:22:42 +08:00 |
|
|
|
3f1d40960f
|
[KV Offload] Fix num_tokens_after_batch for different termination types (#49285)
Signed-off-by: Alex <jihuihuang@example.com>
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: Alex <jihuihuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 16:22:58 +03:00 |
|
|
|
5559679229
|
[Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks (#49052)
Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: jasl <jasl9187@hotmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 14:53:32 +03:00 |
|
|
Chang GuoandOpenAI Codex
|
7eca0e1a64
|
[KV Offload] Deduplicate replicated MLA KV in the shared CPU region (#48906)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-26 08:22:30 +03:00 |
|
|
Rui YinandOr Ozeri
|
d30b1ecd1b
|
[Bugfix][KV Offloading] Defer request finalization until final store (#49671)
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-25 20:58:09 +00:00 |
|
|
Yifan QiaoandClaude
|
70052fb924
|
[Bugfix][KV Connector][Mooncake] Keep TP-sharded Mamba state out of the KV-head dedup (#49499)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-24 21:02:22 -07:00 |
|
|
Nicolò Lucchesiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
589a5b884b
|
[PD][NixlPush][Bugfix] Fix blocking handshake call on writer thread (#49221)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-24 01:41:37 -07:00 |
|
|
|
a76df87db8
|
[MooncakeStore] Re-derive full external hits on stored boundaries (#49481)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-23 18:12:28 +08:00 |
|
|
|
a4904ba903
|
[Perf][KVConnector][Mooncake] Vectorize prepare_value on the KV load path (#48531)
Signed-off-by: girasoley <girasoley@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: girasoley <girasoley@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-23 18:12:00 +08:00 |
|
|
Ziming Huang
|
7c21548ce3
|
[PD][Bugfix] Fix NIXL hybrid MLA+mamba heterogeneous TP (#49297)
Signed-off-by: ZeldaHuang <zelda.huanghuang@gmail.com>
|
2026-07-22 07:11:39 +00:00 |
|
|
|
6700813f86
|
[3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions (#44456)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-21 09:16:15 +00:00 |
|
|
Chauncey
|
94ed0bf4e0
|
[Bugfix][KV Offloading] Handle queued request aborts without allocated KV blocks (#49146)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-07-21 11:16:26 +08:00 |
|
|
|
fbfe58133d
|
[Bugfix][KV Offload] Preserve reachable tails for hybrid SWA groups (#48911)
Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-20 22:12:36 +03:00 |
|
|
Chang GuoandOpenAI Codex
|
f007cceb42
|
[KV Offload] Support self-describing KV events with TieringOffloadingSpec (#48679)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-20 16:41:58 +03:00 |
|
|
Sihan ChenandLi, Jiang <jiang1.li@intel.com>
|
4938d44a3b
|
[CPU] fixes heterogeneous NIXL KV transfer into CPU_ATTN decode workers (#47871)
Signed-off-by: Spycsh <sihan.chen@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-07-20 07:33:13 +00:00 |
|
|
|
b6ff8a2f50
|
[Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-19 02:53:15 +00:00 |
|
|
AlexHuangandOr Ozeri
|
f38f3d11fb
|
[Bugfix][KV Offloading] Offload last block at request finish and prevent reuse race (#48596)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-17 16:50:49 +03:00 |
|
|
|
426d48bfa1
|
[KV Offload] Add optional tier locality to FS/OBJ KV events (#48281)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-17 10:31:52 +03:00 |
|
|
Debasish MohantyandOr Ozeri
|
472d330c21
|
Add blocks_per_chunk configuration for KV offloading to support heterogeneous KV cache groups (#48878)
Signed-off-by: Debasish-87 <22btics06@suiit.ac.in>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-17 09:00:12 +03:00 |
|
|
|
971dac2caa
|
[Bugfix][KV-transfer] MoRIIO: retry RDMA send-queue-full backpressure instead of failing the read (#47495)
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
Signed-off-by: Edwin Lim <edwinlim0919@gmail.com>
Signed-off-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: harishk-mangoboost <harish.kambhampaty@mangoboost.io>
|
2026-07-16 20:02:27 +00:00 |
|
|
Nicolò Lucchesi
|
a317bc5739
|
[Misc][Nixl] Unify _logical_to_remote_kernel_block_ids (#48717)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-16 18:36:25 +08:00 |
|
|
|
a9531edfa6
|
[KV Offload] Define clean backend configuration boundary (#48150)
Signed-off-by: Change72 <changg@nvidia.com>
Signed-off-by: Chang Guo <cguo51@asu.edu>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-16 13:27:05 +03:00 |
|
|
Ilia Yastrebov
|
9f8cbfd8eb
|
Vectorize prep xfer list creation (#48209)
Signed-off-by: Ilia Yastrebov <iyastrebov@nvidia.com>
|
2026-07-16 11:39:05 +02:00 |
|
|
Elvir Crnčević
|
12f2c515a7
|
[Bugfix] Fix offloading set_ overflow for packed non-uniform KV caches (#48530)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
|
2026-07-16 10:04:39 +03:00 |
|
|
Nicolò Lucchesi
|
66b6c684ab
|
[PD][Bugfix] Fix validation of cache shape for attn backends enforcing different kernel_block_size (#48125)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-15 18:26:02 +08:00 |
|
|
|
32aef44388
|
[Bugfix] Include inline per-token-head scales in offloaded page transfer width (#48411)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Itay Etelis <Itay.etelis@gmail.com>
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-07-14 16:07:26 +03:00 |
|
|
tomerg-nvidiaandNickLucche
|
7a74a9662b
|
[NIXL] Avoid reading expired blocks in bidirectional turn-2 read (#47021)
Signed-off-by: Tomer Gilad <tgilad@nvidia.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-14 13:03:41 +00:00 |
|
|
Nick Hill
|
8ac8375270
|
[Core] Preserve Marconi caching with selective hybrid cache retention (#47782)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-13 21:24:20 +01:00 |
|
|
AlexHuang
|
4c81772e8b
|
[Bugfix][KV Offloading] Fix stale transfer_jobs after reset_cache + harden job completion (#48102)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
|
2026-07-12 20:00:04 +03:00 |
|
|
Jiangyun ZhuandYifan Qiao
|
481e481be7
|
[2/N][Core] support partial prefix cache hit for hybrid model (#46384)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-07-12 05:37:51 +00:00 |
|
|
|
51878e5b6e
|
[2/N][KV-Cache Layout Refactor] Pack K/V into the content dim across attention backends (#44455)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2026-07-11 11:11:16 -04:00 |
|
|
|
2ded1b24e7
|
[KV Connector][Mooncake] Apply SWA lookup mask before hashing/key build (#47317)
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-09 19:51:23 +00:00 |
|
|
weishu
|
2285cfca46
|
[KVConnector] MultiConnector: give every sub-connector the request's real blocks in update_state_after_alloc (#46865)
Signed-off-by: deng451e <838677410@qq.com>
|
2026-07-09 11:10:39 -07:00 |
|
|
qizixiandNick Hill
|
7802c20c4e
|
[KVConnector][NIXL] Support pipeline-parallel prefill in push mode (#45880)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-08 16:49:23 -07:00 |
|
|
|
3354dba381
|
[Bugfix][KV offload] Store interior chunk-boundary blocks under MTP/Eagle (#46972)
Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 13:16:52 +03:00 |
|
|
|
e040899a00
|
[KV Offloading] Add basic offloading metrics (#45958)
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Signed-off-by: Srinivas Krovvidi <194645829+Srinivasoo7@users.noreply.github.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-07 09:26:28 +03:00 |
|