|
c5d840ff6a
|
[KV Connector][Offloading] Certify attention-only hybrids in the canonical portability gate (#51689)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-08-31 15:53:19 +03:00 |
|
|
Moein Khazraee
|
9acbc5360a
|
[KV Offload] Preserve KV event metadata until final residency removal (#52068)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
|
2026-08-31 13:28:30 +03:00 |
|
|
|
f9d666f917
|
[KV Offload] Forward ownership in KV cache events (#52067)
Signed-off-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-08-31 10:28:41 +03:00 |
|
|
+5
|
e126687a9a
|
[Model] Support Qwen3.8-Flash-Next (#53896)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-31 05:57:56 +00:00 |
|
|
Isotr0py
|
555ea65e8c
|
[Frontend] Add video embeds input support (#54242)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-08-31 03:20:47 +00:00 |
|
|
AI-Infra Rookie
|
c92b29a1d4
|
[Bugfix] Make metadata send non-blocking in GroupCoordinator.isend_tensor_dict (#49274)
Signed-off-by: z00897177 <zhouxinyi6@huawei.com>
|
2026-08-30 23:51:25 +00:00 |
|
|
Nicolò Lucchesi
|
7f4793eaa3
|
[Nixl][PD] DCP support for MLA models (#50611)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-29 16:03:53 +00:00 |
|
|
|
a758a9f671
|
[Perf][KV Connector] Pin token_indices before non_blocking H2D in hf3fs helper (#54293)
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
2026-08-28 22:20:17 -07:00 |
|
|
|
6b110badbb
|
[Bugfix][Mooncake] Save exact Mamba boundary states (#51358)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Dao Le <daole@inferact.ai>
|
2026-08-29 02:40:53 +00:00 |
|
|
Wei ZhaoandYifan Qiao
|
7fd9cc036e
|
[KV Connector] Support MooncakeStore with hybrid DCP prefix caching (#53324)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: wzhao18 <weizha@nvidia.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-29 02:10:23 +00:00 |
|
|
|
99013d77d3
|
[Bugfix][Distributed] Gate cross-node MNNVL custom all-reduce by group capability (#53253)
Signed-off-by: JulianZJN <148066842+JulianZJN@users.noreply.github.com>
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
|
2026-08-29 08:49:24 +08:00 |
|
|
Xuan-1998andVladimir Aerov
|
026d5af7f4
|
[Bugfix] Fix ncclCommQueryProperties heap overflow with NCCL >= 2.31 (#53008)
Signed-off-by: Xuan Jiang <xuanj@amazon.com>
Co-authored-by: Vladimir Aerov <vaerov@amazon.com>
|
2026-08-29 00:25:00 +00:00 |
|
|
Summer YangandNick Hill
|
2aac565cae
|
[Core][KV Connector] Start async KV loads after the forward launch when no sync loads are scheduled (#53333)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-28 21:58:00 +00:00 |
|
|
Matthew Bonanni
|
d85708f7a4
|
[1/N][KV Connector] Identify externally transferable KV cache groups (#53779)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-08-27 11:21:24 -04:00 |
|
|
Khushali Desai
|
c7b0467c25
|
buffer size insuffient Dspark sd for FlashInfer MNNVL allreduce (#50932)
|
2026-08-27 11:01:08 -04:00 |
|
|
|
6c4aa8bec3
|
[RL] Support checkpoint-coordinate sparse NCCL weight updates (#53751)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-27 18:19:51 +08:00 |
|
|
omerpaz95andClaude Opus 5
|
f8d38fbc87
|
[EC Connector] EC Offloading Connector use events instead of StepTracker (#49994)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-27 03:41:46 +00:00 |
|
|
Avinash Paulandavininjamay8
|
8d301f075b
|
[Bugfix][ROCm][Disagg] Fix MoRIIO shared KV memory region registration (#53698)
Signed-off-by: avininjamay8 <avpaul@amd.com>
Co-authored-by: avininjamay8 <avpaul@amd.com>
|
2026-08-27 00:10:57 +08:00 |
|
|
Artem Perevedentsev
|
2cd6c664c1
|
[MoE] enable all2all fi_one_sided by default (#53311)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
|
2026-08-26 17:03:45 +05:00 |
|
|
Ziming HuangandOpenAI Codex
|
f14369c7f9
|
[Bugfix][Mooncake] Fix Mamba prefill truncation ordering (#53663)
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-26 01:08:37 -07:00 |
|
|
|
ee5ae2ae70
|
[RL] Add rank-local IPC weight updates (#52497)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-08-26 15:36:57 +08:00 |
|
|
Jee Jee Li
|
88e1b11313
|
[Perf] Tune FlashInfer all-reduce thresholds for single-node TP8 on SM103 (#53606)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-08-26 01:46:56 +00:00 |
|
|
Ziming HuangandRoger Wang
|
d4c4ceb31e
|
[Bugfix][NIXL] Fix Mamba prefill truncation ordering (#53523)
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-08-24 19:13:27 -07:00 |
|
|
Quentin Gallouédec
|
26858770ec
|
[Bugfix] Reuse CUDA streams in packed weight transfer to cap reserved-memory waste (#52951)
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
|
2026-08-23 23:54:41 -07:00 |
|
|
Summer Yang
|
e6e1af4ca1
|
[Perf] Tune FlashInfer all-reduce selection on SM103 (#53318)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
|
2026-08-24 05:19:59 +00:00 |
|
|
Aaron Hao
|
6cd9713463
|
[RL] P2P RDT weight sync (#43375)
|
2026-08-23 09:42:22 -07:00 |
|
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
0a3a4f7f35
|
[Bugfix][KV Connector][NIXL] Support PCP producers (#52779)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-21 16:22:01 -04:00 |
|
|
shijuzhaoandshijuzhao
|
cda3868f5d
|
[Bugfix] Deterministic MoE combine (reduce_scatterv) under VLLM_BATCH_INVARIANT (#45683)
Signed-off-by: shijuzhao <shijuzhao@tencent.com>
Co-authored-by: shijuzhao <shijuzhao@tencent.com>
|
2026-08-21 21:41:31 +08:00 |
|
|
Harish Gaikwad
|
c8438a3d40
|
[Doc] Fix dead link in KV transfer README (#53230)
Signed-off-by: hagaikwa-redhat <hagaikwa@redhat.com>
|
2026-08-21 12:03:41 +00:00 |
|
|
Ziming Huang
|
5b1e7a812b
|
[BugFix][Mooncake] Fix Mooncake saves from sparse Mamba block tables (#51362)
Signed-off-by: ZeldaHuang <zelda.huanghuang@gmail.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
|
2026-08-20 03:05:43 -07:00 |
|
|
Woosuk KwonandOpenAI Codex
|
6a962071bd
|
[Distributed] Enable FlashInfer all-reduce by default (#52998)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-19 18:17:43 -07:00 |
|
|
Guanyi Chenandz-zanez
|
bf2866f8bf
|
[KV Connector] Add decode offloading to Mooncake Store consumers (#52466)
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Guanyi Chen <939416532@qq.com>
Co-authored-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
|
2026-08-19 17:40:35 -07:00 |
|
|
Chaojun Zhang
|
f485081e8b
|
[XPU] Support EC connector KV Offloading on XPU (#49532)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
|
2026-08-19 10:25:23 +08:00 |
|
|
Itay Alroy
|
a2257f95b7
|
[Elastic EP] Reduce eager-mode reconfiguration downtime (#51885)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-08-19 02:13:06 +00:00 |
|
|
Bo Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c296851a7d
|
[MoE] Refine FlashInfer one-sided All2All integration (#51924)
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-18 00:18:14 +00:00 |
|
|
Tyler Michael SmithandOpenAI Codex
|
017e9f4448
|
Promote prefix_cache_retention_interval to an argument and change the default to 0 (#52216)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-17 13:20:50 +00:00 |
|
|
Guanyi Chen
|
dc9ae4b8ac
|
[Bugfix][Mooncake] Reference GPU blocks for in-flight store jobs and key the store ledger by store_job_id (#52372)
Signed-off-by: Guanyi Chen <939416532@qq.com>
|
2026-08-16 16:09:46 -07:00 |
|
|
|
fa9d67f782
|
[EC Connector] Added Build Connector Worker Meta for EC Connector (#49585)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-08-15 15:40:38 -07:00 |
|
|
Nick HillandClaude Opus 4.8
|
615d4cfade
|
[Core] Check for GPU<->CPU syncs during CI (#43107)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 17:42:05 -07:00 |
|
|
Tzu-Ling KanandClaude Opus 4.8
|
653cc6faca
|
[Bugfix][NIXL] Include transfer mode (push/pull) in the compatibility hash (#50620)
Signed-off-by: Tzu-Ling <tzulingk@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-14 12:11:22 +08:00 |
|
|
Ziqi Fan
|
15227b934a
|
[Bugfix][KV Offload] Emit self-describing CPU events at KV-group block granularity (#51614)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2026-08-12 20:28:15 -07:00 |
|
|
Ziqi Fan
|
f9a0f629bd
|
[KV Offload] Expose data-parallel topology to offloading backends (#51879)
Signed-off-by: Ziqi Fan <ziqif@nvidia.com>
|
2026-08-12 20:27:26 -07:00 |
|
|
Julien Debache
|
6e502b62cb
|
fix and test EPLB balancedness calculation (#51813)
Signed-off-by: jdebache <jdebache@nvidia.com>
|
2026-08-12 20:27:22 -07:00 |
|
|
Aaron HaoandClaude Opus 5
|
793ca6998a
|
[Docs][RL] Rewrite weight-transfer docs; standardize examples (#51729)
Signed-off-by: haoaaron <ahao@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-11 18:53:15 -07:00 |
|
|
liuzhenweiandKunshang Ji
|
d648236199
|
[XPU][CI] Fix ExampleConnector KV cache device selection (#51806)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-08-11 23:40:29 +08:00 |
|
|
Itay EtelisandItay Etelis
|
0f2ea973be
|
[KV Connector][Offloading] Keep per-layer KV registration when canonical_layout is requested (#51688)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-08-11 06:03:59 +03:00 |
|
|
acmore
|
f37dff22fc
|
[Bugfix] Import each packed IPC export once on the consumer side (#51259)
Signed-off-by: acmore <hezj07+github@gmail.com>
|
2026-08-11 10:05:27 +08:00 |
|
|
Itay EtelisandItay Etelis
|
81840a172f
|
[KV Connector] Canonical CPU layout for parallelism-agnostic KV offload (#48414)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
|
2026-08-10 10:44:54 +03:00 |
|
|
|
11ba93f364
|
[BugFix] Use file:// rendezvous for single-node executors to eliminate startup port races (#50999)
Signed-off-by: Parsa Mazaheri <p_mazaheri@apple.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Parsa Mazaheri <p_mazaheri@apple.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-09 22:00:04 -04:00 |
|