 Wei Zhao
|
5fd3b276f8
|
[Mooncake] Skip KV lookup for non-reachable SWA blocks (#45444)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-06-18 02:23:20 -07:00 |
|
|
AlexHuangandOr Ozeri
|
421c1ec448
|
[KV Offloading] Remove dummy worker-side stats from OffloadingConnector (#45905)
Signed-off-by: Alex <alex.tech.lab@outlook.com>
Signed-off-by: AlexHuang <jihuihuang@alexai.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-18 08:13:28 +03:00 |
|
|
Ronen Schaffer
|
f428718ffe
|
[Fix][KV offload] Defer on_request_finished until in-flight transfers drain (#45823)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-18 07:05:46 +03:00 |
|
|
Jonathan Chenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d57888efa4
|
[SimpleCPUOffloadConnector]: Add support for reset_cache() (#39726)
Signed-off-by: Jonathan Chen <chenleejonathan@gmail.com>
Signed-off-by: Jonathan <chenleejonathan@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 19:47:12 -07:00 |
|
|
Bryan Shanandwaynehacking8
|
0d339cf135
|
[Bugfix] Fix NixlConnector handshake block_len validation for GQA-replicated KV heads (#45879)
Signed-off-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
Co-authored-by: waynehacking8 <waynehacking8@gmail.com>
|
2026-06-17 15:11:29 -07:00 |
|
|
shanjiazandshanjiaz
|
5fd21eb0b2
|
[BUG] fix hidden states nan for hybrid attention models (#45849)
Signed-off-by: shanjiaz <hezhao@redhat.com>
Co-authored-by: shanjiaz <hezhao@redhat.com>
|
2026-06-17 18:02:24 -04:00 |
|
|
danisereb
|
5e27b2baf4
|
[Bugfix] Pass TP group to FlashInfer all-reduce fusion (#45917)
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
|
2026-06-17 15:24:26 +00:00 |
|
|
zhanqiuhu
|
eb0fdeb1e8
|
[Bugfix][PD] Fix DSV4 disaggregated serving (#45831)
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
|
2026-06-17 15:17:14 +00:00 |
|
|
|
6d8fff5698
|
[KV Connector][Offloading] Avoid blocking the engine to flush offloads on idle (#45595)
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Signed-off-by: Itay Etelis <Itay.etelis@gmail.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Itay Etelis <Itay.etelis@gmail.com>
|
2026-06-17 11:35:07 +03:00 |
|
|
Dao007foreverandClaude Opus 4.8
|
44b2512767
|
[KV Connector][Mooncake] Add cache_prefix to namespace store keys (#45767)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-16 10:24:20 -07:00 |
|
|
kourosh hakhamaneshiandClaude Opus 4.8
|
188c68798e
|
[KVConnector][MoRIIO] Allow overriding the advertised host IP (#45488)
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-16 17:18:37 +00:00 |
|
|
Hank Han
|
d53f4593ce
|
[KV Connector][Mooncake] Pipeline-parallel support for PD-disaggregated serving with Mooncake connector (#44528)
Signed-off-by: hanhan.hank <hanhan.hank@bytedance.com>
Signed-off-by: Hank Han <hanhan7630@outlook.com>
|
2026-06-16 04:35:38 -07:00 |
|
|
|
7ad894c86a
|
[Bugfix] Prevent cuMemcpyBatchAsync segfault with MTP and KV offloading (#44784)
Signed-off-by: joshua <joshua.abraham@multicorewareinc.com>
Co-authored-by: joshua <joshua.abraham@multicorewareinc.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-16 07:58:39 +00:00 |
|
|
snadampalandNicolò Lucchesi
|
88ed636218
|
[KV Connector]: Support KV push from Prefill to Decode node using Nixl KV Connector (#35264)
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2026-06-12 10:38:41 +00:00 |
|
|
Dao007forever
|
6fbfdd1831
|
[NIXL] Per-region KV transfer classification for mixed full-attn + MLA groups (#44583)
|
2026-06-11 21:42:41 -07:00 |
|
|
Varun Sundar Rabindranathandvarun sundar rabindranath
|
b927004c44
|
[Bugfix] Mamba CPU Offloading (#44599)
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
|
2026-06-11 21:07:35 -07:00 |
|
|
|
4bc83323f2
|
[Bugfix] OffloadingConnector: respect skip_reading_prefix_cache flag (#44592)
Signed-off-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
Signed-off-by: littlecircle0730 <43994952+littlecircle0730@users.noreply.github.com>
Co-authored-by: Hsiao-Yuan Chen <hy.c@Hsiao-YuandeMacBook-Pro.local>
Co-authored-by: Or Ozeri <or@ozery.com>
|
2026-06-12 02:20:39 +00:00 |
|
|
Dao007foreverandClaude
|
8a91228dbe
|
[Bugfix][KVConnector][Mooncake] Close MooncakeDistributedStore on connector teardown (#45206)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-11 14:33:48 -07:00 |
|
|
fangyuchuandTyler Michael Smith
|
23eb7c8fbb
|
[Bugfix] Fix NixlEPAll2AllManager's dependency on --enable-elastic-ep to function (#44422)
Signed-off-by: fangyuchu <fangyuchu@qq.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2026-06-11 08:14:49 -07:00 |
|
|
Nicolò Lucchesi
|
750aab5b8e
|
[Bugfix] Fix CPU memory leak related to not cleaning up old remotes data (#44424)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-11 07:54:52 -07:00 |
|
|
Fynn Schmitt-Ulmsandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ebc6ef971a
|
Hidden states extraction improvements (#43805)
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-11 09:44:45 -04:00 |
|
|
Yifan Qiao
|
f272dfdce1
|
[KV Connector] Mooncake store: prefix-cache retention interval for sparse attention (#44774)
|
2026-06-10 21:36:34 -07:00 |
|
|
Ilya Markov
|
6471ec75bd
|
[EPLB] Reject NCCL-based EPLB communicators with async EPLB (#44978)
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
|
2026-06-10 19:51:27 +00:00 |
|
|
|
9dfc313bdc
|
Feature/offloading manager stats (#35669)
Signed-off-by: Sriusa4414@gmail.com
Signed-off-by: srinivas_oo7 <Sriusa4414@gmail.com>
Signed-off-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Signed-off-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: srinivas_oo7 <sklinkedin0120@gmail.com>
Co-authored-by: Srinivasoo7 <158864704+Srinivasoo7@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-10 12:44:55 +00:00 |
|
|
Dao007foreverandClaude Opus 4.7
|
c1d754d681
|
[Mooncake] Use all HCAs on multi-NIC hosts instead of GPU-indexed RNIC selection (#43799)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-06-09 11:05:36 -07:00 |
|
|
Nicolò Lucchesi
|
6690a0c4de
|
[PD][Bugfix] Fix KV Cache sharing with HMA (#44629)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-09 06:10:06 -07:00 |
|
|
Sage
|
5b3807e862
|
[KV Events] Switch event structs from array to map encoding (#42892)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
|
2026-06-09 11:39:52 +00:00 |
|
|
Tyler Michael SmithandClaude Opus 4.6
|
e2f993dc41
|
[WideEP] Integrate DeepEP v2 (#41183)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-06-08 18:07:29 -07:00 |
|
|
bnellnm
|
dc68bd8c41
|
[MoE Refactor] FusedMoE/MoERunner inversion refactor (#41184)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-06-08 10:42:58 -04:00 |
|
|
Nicolò Lucchesi
|
5add018beb
|
[Connector] Remove P2pNcclConnector (#44854)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-08 18:58:29 +08:00 |
|
|
Yifan Qiao
|
51ef688831
|
[Bugfix][Mooncake] Fix per-group block_size/block_hash and group_idx in MooncakeStoreConnector KV events (#44103)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-07 07:12:43 +00:00 |
|
|
Harry Mellor
|
a80af24356
|
Speed up docs build (#44635)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-05 14:51:44 +00:00 |
|
|
Nicolò Lucchesi
|
d98b8f371c
|
[NixlConnector] Initiate deprecation cycle for kv_both role (#43874)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-05 11:08:17 +02:00 |
|
|
qizixi
|
96229fa99e
|
[KVConnector][1/N] PP-aware handshake aggregation and intermediate-PP output plumbing (#43720)
Signed-off-by: zixi-qi <zixi@inferact.ai>
|
2026-06-04 22:04:19 -07:00 |
|
|
Wentao Ye
|
41a4829f22
|
[Logs Refactor] Optimize shutdown logs, easier to follow and consistent (#43707)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-06-04 14:36:32 -04:00 |
|
|
Tushar JainandTushar Jain
|
38fd2405f3
|
use split_group for pytorch process group creation (#41980)
Signed-off-by: Tushar Jain <tushar00jain@users.noreply.github.com>
Co-authored-by: Tushar Jain <tushar00jain@users.noreply.github.com>
|
2026-06-04 14:36:07 -04:00 |
|
|
Nicolò Lucchesi
|
68f5e565c9
|
[PD][Nixl] Mamba prefix caching mode support (#42554)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2026-06-04 06:41:46 -07:00 |
|
|
|
4f423bd5bc
|
[EPLB] Nixl communicator optimization. Zero-copy transfers (#41633)
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-06-04 03:40:34 +00:00 |
|
|
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
e6018c644a
|
[Refactor] Remove dead code in tests and parallel_state (#41471)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-03 19:32:39 -07:00 |
|
|
pschlan-amdandShengqi Chen
|
0a5cbf633e
|
Handle spinloop ext load failure gracefully (#43659)
Signed-off-by: Patrick Schlangen <pschlan@amd.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-06-03 16:09:52 +00:00 |
|
|
|
0c6631f02a
|
[KVCache] Support Pluggable KVCacheSpec (#37505)
Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-03 09:05:16 -07:00 |
|
|
Nick Hill
|
e15f20258b
|
[ModelRunnerV2] Avoid pipeline parallel bubbles (#42187)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-02 14:02:01 -07:00 |
|
|
Yifan Qiao
|
e9e08c49b9
|
[Bugfix] Cache the EAGLE/MTP lookahead block in the SWA prefix-cache mask (#44082)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-06-02 12:21:07 -07:00 |
|
|
Wei ZhaoandWei Zhao
|
2427094152
|
[Feature] Support EPLB for DeepSeek v4 Mega Moe (#43339)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-lyris01.lyris.clusters.nvidia.com>
|
2026-06-02 10:56:44 -07:00 |
|
|
Ronen Schaffer
|
2a2b5ca791
|
[KV Offload] Add on_schedule_end() hook to separate step lifecycle from event draining (#44206)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-06-02 13:42:52 +03:00 |
|
|
|
689b0eeb9e
|
[HARDWARE][POWER] Enable SHM communicator support for PowerPC (#43754)
Signed-off-by: Rukhaiya <rukhaiya@c643n08aix1-lp1.pok.stglabs.ibm.com>
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Rukhaiya <rukhaiya@c643n08aix1-lp1.pok.stglabs.ibm.com>
Co-authored-by: Akash kaothalkar <61960177+Akashcodes732@users.noreply.github.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-06-02 18:06:32 +08:00 |
|
|
omerpaz95andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
d247a9dc13
|
[EC Connector] Non blocking EC Connector lookup (#41627)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-02 08:48:25 +00:00 |
|
|
Or Ozeri
|
480fadab1b
|
[BugFix][kv_offload]: Prevent offloading stale sliding window blocks (#42959)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
|
2026-06-02 05:59:48 +03:00 |
|
|
Dao007foreverandClaude
|
d68f0b220e
|
[Bugfix][Mooncake] Release GPU pin on failed store in MooncakeStoreConnector (#43742)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-01 18:29:18 -07:00 |
|
|
Siddharth BedekarandOpenAI Codex
|
266b9d9c64
|
[Frontend][Core] Add sparse NCCL weight transfer support for in-place updates (#40096)
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-06-01 15:37:30 -04:00 |
|