 Aaron Hao
|
30b4e7f479
|
[rl] Stateful Trainer Send: IPC [2/N] (#48981)
Signed-off-by: haoaaron <ahao@anyscale.com>
|
2026-07-30 21:59:08 +08:00 |
|
|
+13
|
aeeb36b1f1
|
[New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-07-30 03:49:58 -07:00 |
|
|
|
4e582c5b54
|
[PD][Bugfix] Rebase KV lease deadlines onto worker clock (#50326)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-07-30 01:25:42 +00:00 |
|
|
Nick HillandClaude Opus 5
|
437e0b7f8e
|
[BugFix] Fix P/D preemption race condition (#50297)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-07-30 01:11:18 +00:00 |
|
|
bugkeepandClaude
|
48aa8d8d75
|
[Bugfix] Prevent stale multiproc RPC deadlines from becoming unbounded waits (#41357)
Signed-off-by: bugkeep <1921817430@qq.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-29 16:41:34 -07:00 |
|
|
|
5fa0154448
|
[Rubin] Enable NVLink all-reduce paths on SM107 (#49647)
Signed-off-by: Zachary Aristei <zaristei@nvidia.com>
Co-authored-by: Sahithi Chigurupati <chigurupati.sahithi@gmail.com>
Co-authored-by: hlu1 <14827759+hlu1@users.noreply.github.com>
|
2026-07-29 20:53:25 +00:00 |
|
|
Connor CarpenterandNick Hill
|
48fc2e2797
|
feat(grpc): add KV event source discovery (#50033)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-29 13:46:40 -07:00 |
|
|
Nick Hill
|
82553692b8
|
[KV Connector] Fix NIXL mamba state pairing for multi-slot block tables (#50153)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-29 09:45:53 -07:00 |
|
|
omerpaz95
|
dad7a6383b
|
[EC Connector] Add has_pending_push_work (#49582)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
|
2026-07-29 11:04:22 +02:00 |
|
|
Nick HillandJared Wen
|
f37f03db4a
|
[KV Connector] Support NIXL P/D for hybrid MLA+SSM models (#49762)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Jared Wen <w13431838023@gmail.com>
|
2026-07-29 11:50:30 +08:00 |
|
|
Ruinan MaandNick Hill
|
a07fac758f
|
[Perf] Zero-copy torch.Tensor pickling in shm_broadcast MessageQueue (#48442)
Signed-off-by: Ruinan Ma <r7ma3088@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-28 16:28:28 -07:00 |
|
|
Jonguk CheongandOpenAI Codex
|
1e81853afc
|
[Bugfix][KV Offload] Keep Mamba block span unscaled under DCP (#49964)
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-28 17:41:55 +03:00 |
|
|
Nick HillandClaude Fable 5
|
601fa9a74e
|
[KV Connector] Support NIXL heterogeneous P/D block sizes for hybrid models (#49612)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-28 06:45:02 -07:00 |
|
|
Nicolò Lucchesi
|
98e91a9600
|
[PD][NixlPush] Skip extra add_remote_agent step in D->P handshake (#49345)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-28 14:57:12 +02:00 |
|
|
Itay Alroy
|
35efdf6b34
|
[Elastic EP] Async preparation (#47288)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-07-28 05:19:13 -07:00 |
|
|
Shuolei Wang
|
9069a57139
|
[Core][Frontend] Add weight version tagging for RL rollouts (#49040)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
|
2026-07-28 14:35:29 +08:00 |
|
|
|
1206891822
|
[ROCm][KVConnector][MoRI-IO] Fix WRITE-mode remote-TP rank collapse (#46332 follow-up) (#47764)
Signed-off-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com>
Co-authored-by: avininjamay8 <avininjamay8@users.noreply.github.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
|
2026-07-27 20:26:24 -05:00 |
|
|
|
831d3848f1
|
[Core] Fail fast when /dev/shm is too small for the shm ring buffer (#48879)
Signed-off-by: Dr Andrea Tassi <andrea@verticular.uk>
Co-authored-by: Dr Andrea Tassi <andrea@verticular.uk>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-27 19:48:35 +00:00 |
|
|
Rishi PuriandClaude Opus 4.8
|
fd10e8946d
|
[Test] Regression test for hybrid-Mamba eagle cache-peek in Mooncake connector (#43559) (#48361)
Signed-off-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-27 19:03:58 +00:00 |
|
|
Nicolò Lucchesi
|
2b465b2c42
|
[Misc][PD] Nixl cleanup get_backend_aware_kv_block_len and virtually_split_kv_in_blocks (#49988)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-27 18:51:37 +02:00 |
|
|
rongfu.lengandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
81962bb699
|
[Bugfix]Reject invalid FlashInfer MNNVL workspaces (#49043)
Signed-off-by: lengrongfu <lenronfu@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-27 08:12:23 -04:00 |
|
|
Ronen Schaffer
|
77cba0259f
|
[KV Offloading] Per-request tier filtering with TierFilter/TierMatcher (#48123)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-07-27 13:29:57 +03:00 |
|
|
Dao007foreverandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d742856610
|
[3/N][Core][KV Connector] Support reliable partial-tail KV offload for sub-block prompts (#49502)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-26 23:22:17 -07:00 |
|
|
limeward
|
ffc4f08c8e
|
[Core][KV-transfer] MoRIIO: heterogeneous TP<->DP prefill/decode read routing (#46116)
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
|
2026-07-27 02:10:00 +00:00 |
|
|
Schwinn Saereesitthipitak
|
b5b61c622c
|
[Core][Distributed] Add process-checkpoint lifecycle hooks for communicators (starting with Flashinfer) (#46877)
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
|
2026-07-26 14:47:50 -04:00 |
|
|
|
b68d7ef262
|
[Bugfix][KV Offload] Namespace auto cache dtype by effective dtype (#49438)
Signed-off-by: Jonguk Cheong <jdal3031@snu.ac.kr>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 20:59:09 +03:00 |
|
|
Daniel SocekandKunshang Ji
|
7154856f3d
|
[Bugfix] Fix handling 5D KV cache in kv_postprocess_layout_on_receive (#47791)
Signed-off-by: Daniel Socek <daniel.socek@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-26 22:22:42 +08:00 |
|
|
|
3f1d40960f
|
[KV Offload] Fix num_tokens_after_batch for different termination types (#49285)
Signed-off-by: Alex <jihuihuang@example.com>
Signed-off-by: Alex <jihui.huang@daocloud.io>
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: Alex <jihuihuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 16:22:58 +03:00 |
|
|
|
5559679229
|
[Bugfix][KV Offload] Bound unaligned SWA loads by physical GPU blocks (#49052)
Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: jasl <jasl9187@hotmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-26 14:53:32 +03:00 |
|
|
Chang GuoandOpenAI Codex
|
7eca0e1a64
|
[KV Offload] Deduplicate replicated MLA KV in the shared CPU region (#48906)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-26 08:22:30 +03:00 |
|
|
Rui YinandOr Ozeri
|
d30b1ecd1b
|
[Bugfix][KV Offloading] Defer request finalization until final store (#49671)
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-25 20:58:09 +00:00 |
|
|
|
0b0bd2b5f6
|
[Feature] Add fault tolerance framework (simplified) for DP+EP external LB deployments (#44428)
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Signed-off-by: a798347923 <2645302020@qq.com>
Signed-off-by: TianZhuo <2770730562@qq.com>
Signed-off-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Signed-off-by: 205150940 <112750056+205150940@users.noreply.github.com>
Signed-off-by: w00689259 <wangzhuo66@huawei.com>
Signed-off-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Signed-off-by: zWaNg3 <389750525@qq.com>
Signed-off-by: yzchang-plus <1078477584@qq.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: zWaNg3 <37772915+zWaNg3@users.noreply.github.com>
Co-authored-by: a798347923 <2645302020@qq.com>
Co-authored-by: TianZhuo <2770730562@qq.com>
Co-authored-by: 205150940 <112750056+205150940@users.noreply.github.com>
Co-authored-by: a798347923 <39047817+a798347923@users.noreply.github.com>
Co-authored-by: w00689259 <wangzhuo66@huawei.com>
Co-authored-by: zWaNg3 <389750525@qq.com>
Co-authored-by: yzchang-plus <1078477584@qq.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
|
2026-07-25 11:49:09 -04:00 |
|
|
Yifan QiaoandClaude
|
70052fb924
|
[Bugfix][KV Connector][Mooncake] Keep TP-sharded Mamba state out of the KV-head dedup (#49499)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-24 21:02:22 -07:00 |
|
|
|
2279575cd9
|
[AMD][Bugfix][EPLB] Fix elastic EP scaling accuracy on ROCm (#47206)
Signed-off-by: okorzh <okorzh@amd.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-24 13:35:27 -07:00 |
|
|
|
5d8e90a966
|
[WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@vllm.ai>
Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-07-24 13:00:02 -07:00 |
|
|
Nicolò Lucchesiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
589a5b884b
|
[PD][NixlPush][Bugfix] Fix blocking handshake call on writer thread (#49221)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-24 01:41:37 -07:00 |
|
|
|
10c75477b0
|
[Bugfix][Core] shm_broadcast: bound idle reader waits and release read slots (#45224)
Signed-off-by: Chaemin Lim <chaemin.lim@mangoboost.io>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Edwin Lim <edwin.lim@mangoboost.io>
Co-authored-by: Jaeyoun Kim <jaeyoun.kim@mangoboost.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-23 18:13:29 +08:00 |
|
|
|
a76df87db8
|
[MooncakeStore] Re-derive full external hits on stored boundaries (#49481)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-23 18:12:28 +08:00 |
|
|
|
a4904ba903
|
[Perf][KVConnector][Mooncake] Vectorize prepare_value on the KV load path (#48531)
Signed-off-by: girasoley <girasoley@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: girasoley <girasoley@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-23 18:12:00 +08:00 |
|
|
Ziming Huang
|
7c21548ce3
|
[PD][Bugfix] Fix NIXL hybrid MLA+mamba heterogeneous TP (#49297)
Signed-off-by: ZeldaHuang <zelda.huanghuang@gmail.com>
|
2026-07-22 07:11:39 +00:00 |
|
|
Andreas Karatzas
|
16aca639b7
|
[ROCm] Upgrade NIXL and UCX (#49251)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-07-21 22:30:02 -05:00 |
|
|
aoshen02andaoshen02
|
ba18929079
|
[Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates (#49178)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
|
2026-07-21 18:56:39 -07:00 |
|
|
|
6700813f86
|
[3/N][KV-Cache Layout Refactor] Standardize Mamba cache; drop get_transfer_cache_regions (#44456)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-21 09:16:15 +00:00 |
|
|
Chauncey
|
94ed0bf4e0
|
[Bugfix][KV Offloading] Handle queued request aborts without allocated KV blocks (#49146)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-07-21 11:16:26 +08:00 |
|
|
|
fbfe58133d
|
[Bugfix][KV Offload] Preserve reachable tails for hybrid SWA groups (#48911)
Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-20 22:12:36 +03:00 |
|
|
Chang GuoandOpenAI Codex
|
f007cceb42
|
[KV Offload] Support self-describing KV events with TieringOffloadingSpec (#48679)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-20 16:41:58 +03:00 |
|
|
Sihan ChenandLi, Jiang <jiang1.li@intel.com>
|
4938d44a3b
|
[CPU] fixes heterogeneous NIXL KV transfer into CPU_ATTN decode workers (#47871)
Signed-off-by: Spycsh <sihan.chen@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-07-20 07:33:13 +00:00 |
|
|
Michał GanczarenkoandKunshang Ji
|
37bf988c2f
|
[XPU][Bugfix] Fix GroupCoordinator device_index (#47295)
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-20 15:25:56 +08:00 |
|
|
|
b6ff8a2f50
|
[Core] Add MRV2 virtual-batch PCP for MLA (#46570)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-19 02:53:15 +00:00 |
|
|
AlexHuangandOr Ozeri
|
f38f3d11fb
|
[Bugfix][KV Offloading] Offload last block at request finish and prevent reuse race (#48596)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-17 16:50:49 +03:00 |
|