Or Ozeri
5ef33ab250
[kv_offload+HMA][10/N]: Support load with multiple KV groups ( #39402 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-23 20:00:45 +03:00
Simon Danielsson and Matvei Pashkovskii
ac58e2a170
[Fix][MoRI] Align MoRI-IO message format with P2pNcclConnector and vllm-router ( #39565 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: Matvei Pashkovskii <mpashkov@amd.com >
2026-04-23 08:06:31 +09:00
Wentao Ye and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
8f87eb4622
[Refactor] Clean up log once scope="local" ( #40540 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-22 16:42:43 -04:00
Or Ozeri and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
aad88f8486
[kv_offload+HMA][8/N]: Support multi-group worker transfer ( #38453 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 08:44:00 +03:00
Soila Kavulya
f90aa44662
[NIXL][XPU]Fix nixl import on XPU ( #40430 )
...
Signed-off-by: Soila Kavulya <soila.p.kavulya.intel.com>
2026-04-22 09:26:33 +08:00
rasmith
cefa5281a7
[ROCm][P/D][MORI][BugFix] Ensure correct api is used when making requests to prefill / decode nodes ( #39835 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
2026-04-22 09:48:25 +09:00
bnellnm
5e584ce9ec
[MoE Refactor] Remove SharedFusedMoE class ( #35782 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-21 18:12:12 -04:00
Sage Moore
3173441b0f
[EPLB] Consolidate is_unchanged/is_received_locally into TransferMetadata ( #37341 )
...
Signed-off-by: Sage Moore <sage@neuralmagic.com >
2026-04-20 21:12:42 +00:00
Cao Qian
8b1f3bebca
[LMCache MP Connector] Add num_lmcache_extra_cached_token in KVTransferParams ( #39843 )
...
Signed-off-by: aeon-x <talexcao@gmail.com >
2026-04-20 20:42:49 +00:00
Tyler Michael Smith
81d954f454
[WideEP] Remove naive all2all. Use allgather_reducescatter instead ( #33728 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-04-20 17:53:55 +00:00
Sage Moore and Tyler Michael Smith
3461c8b027
[EPLB] Refactor Async EPLB synchronization logic ( #37601 )
...
Signed-off-by: Sage Moore <sage@neuralmagic.com >
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com >
2026-04-20 17:05:41 +00:00
zhanqiuhu
cc3993b05d
nixl refactor [2/N]: unify TpKVTopology + HeteroTPTransferConfig into TransferTopology ( #39529 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-20 12:39:08 +02:00
Ilya Markov
50dd4cb427
[EPLB] Add nixl-based eplb communicator ( #36276 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Markov Ilya <markovilya19@gmail.com >
2026-04-20 10:24:23 +00:00
Or Ozeri and Nicolò Lucchesi
f774ba028a
[kv_offload+HMA][4/N]: Support sliding window lookup ( #36645 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-20 12:53:51 +03:00
Fadi Arafeh
2aab9acf48
[CPU][BugFix] Fix inter-node pipeline parallel ( #40150 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-04-20 17:21:12 +08:00
omerpaz95
4353c9cb4a
[KV Offload] Pass request context ( #39185 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
2026-04-19 08:54:59 +03:00
omerpaz95 and Or Ozeri
4b7f5ea1a0
[KV Connector] Allow metrics of multiple connectors of same types in multi connector. ( #40010 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-04-19 07:49:10 +03:00
Chaojun Zhang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
993859ceb0
[XPU] fix all_reduce all-zero accuracy issue under torch.compile ( #39844 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-18 02:33:07 +00:00
zhanqiuhu
41488f2acd
[Bugfix][NIXL] Fix _logical_to_kernel_block_ids conversion for non-mamba models ( #39724 )
...
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
2026-04-15 20:08:58 +00:00
55e1a8e103
[Mooncake] Fix mixed MLA+Eagle block-size validation ( #39596 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-15 11:36:47 -07:00
Roy Huang
ed33310552
[KVConnector][LMCache] Propagate cache_salt through MP connector for per-user cache isolation ( #39837 )
...
Signed-off-by: royyhuang <royyhuang@gmail.com >
Signed-off-by: royyhuang <roy.y.huang@gmail.com >
2026-04-15 09:10:49 -07:00
Or Ozeri
235e1f930a
[kv_offload+HMA][3/N]: Remove block_size from KVEvents ( #36644 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-15 11:53:19 +03:00
Zhewen Li and Zhewen Li
1696c864b9
[Bugfix][Mooncake] Fix thread-local CUDA context for NVLink transfers in _send_blocks ( #39548 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-14 14:13:58 -07:00
maobaolong
b2f749dc97
fix(lmcache): correct store for cached requests while enable prefix cache ( #39719 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
2026-04-14 20:51:27 +00:00
chunxiaozheng and Yihua Cheng
c687bf226a
[LMCache][MP] optimize save when mla enabled ( #38810 )
...
Signed-off-by: idellzheng <idellzheng@tencent.com >
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu >
2026-04-13 17:56:43 -07:00
maobaolong
2a3c32ce67
fix(lmcache): correct store for cached requests and num_scheduled_tokens in lmcache_mp_connector.py ( #39655 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
2026-04-13 03:29:19 +00:00
Nicolò Lucchesi
3244a2ebf2
[KVConnector][NIXL] Organize NIXL connector into its own directory ( #39354 )
...
The number of features supported by the connector has grown substantially
and the `nixl_connector.py` file has accumulated a lot of code. Creates a separate
directory and isolates connector/scheduler code in the hope of improving clarity
and maintainability.
Further refactor of components aimed at improving clarity and simplifying code
will follow soon.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-12 13:10:50 +00:00
Martin Hickey and Or Ozeri
cc07dad789
[HMA] [KVEvent] Enable GPU-side KV events for HMA ( #37688 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-12 10:01:02 +03:00
Ronen Schaffer
a5b17fba8f
[KV Offload] Implement shutdown() in OffloadingConnector and related classes ( #39182 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
2026-04-10 08:06:22 +03:00
Yuwei An and yuwei
467d3247c3
[LMCache] vLLM Block Allocation Event ( #38856 )
...
Signed-off-by: yuwei <yuwei@dev.local >
Co-authored-by: yuwei <yuwei@dev.local >
2026-04-09 17:30:29 -07:00
Chendi.Xue
ef5a226819
[PD][HeteroArch]Fix accuracy issue with CPU_ATTN as Decoder and Flash_ATTN as prefiller ( #38935 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-04-09 11:19:07 +08:00
Or Ozeri
512c5eb455
[kv_offload+HMA][5/N]: Track group block hashes and block IDs ( #37109 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-08 19:50:28 +03:00
Andrey Talman
2111997f96
[release 2.11] Update to torch 2.11 ( #34644 )
2026-04-07 18:55:48 -07:00
ibifrost and Simon Mo
96b5004b71
[KVConnector] Support 3FS KVConnector ( #37636 )
...
Signed-off-by: wuchenxin <wuchenxin.wcx@alibaba-inc.com >
Signed-off-by: ibifrost <47308427+ibifrost@users.noreply.github.com >
Co-authored-by: Simon Mo <simon.mo@hey.com >
2026-04-07 15:46:00 +00:00
maobaolong and Chauncey
729eb59f60
[KVConnector]: prioritize external connector over internal registry ( #38301 )
...
Signed-off-by: baoloongmao <baoloongmao@tencent.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-04-07 15:03:11 +00:00
zhanqiuhu
bfdc0a3a99
[NIXL][Mamba][3/N] Heterogeneous TP: 3-read conv state transfer ( #37635 )
2026-04-06 19:07:02 +02:00
yzong-rh and Nicolò Lucchesi
dc0428ebb8
[NIXL][BUG] Fix Triton heterogeneous TP ( #37940 )
...
Signed-off-by: Yifan <yzong@redhat.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-01 17:23:15 +02:00
bnellnm
7cf56a59a2
[MoE Refactor] Make SharedExperts class for use with DefaultMoERunner ( #35153 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-01 09:44:08 -04:00
yjz and Nicolò Lucchesi
992368522f
[KVTransfer] Fix TpKVTopology.is_kv_replicated equality case ( #38179 )
...
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-01 12:41:49 +02:00
Yifan Qiao
91e4521f9f
[Feat][v1] Simple yet General CPU KV Cache Offloading ( #37160 )
...
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-03-31 17:58:37 -07:00
4dfad17ed1
replace cuda_device_count_stateless() to current_platform.device_count() ( #37841 )
...
Signed-off-by: Liao, Wei <wei.liao@intel.com >
Signed-off-by: wliao2 <wei.liao@intel.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-31 22:32:54 +08:00
Ilya Markov and Markov Ilya
abdbb68386
[EPLB] Add alternative communication for EPLB weight exchange ( #33176 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Markov Ilya <markovilya19@gmail.com >
Co-authored-by: Markov Ilya <markovilya19@gmail.com >
2026-03-31 08:17:12 -04:00
Kfir Toledo and Or Ozeri
6cc7abdc66
[kv_offload+HMA] Fix num_blocks with different per-layer page sizes and improve assert message ( #38554 )
...
Signed-off-by: Kfir Toledo <kfir.toledo@ibm.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-03-31 06:00:40 +00:00
Ilya Markov
12701e8af2
[EPLB] Optmize eplb mapping and record in router for prefill ( #36261 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
2026-03-30 19:48:33 +00:00
Sage Moore
497e234d38
[EPLB] Cleanup the transfer logic for the various eplb maps ( #34520 )
...
Signed-off-by: Sage Moore <sagmoore@redhat.com >
Signed-off-by: Sage Moore <sage@neuralmagic.com >
2026-03-27 10:18:46 +01:00
dtc
6287e7fa20
[P/D] Mooncake: Add unit tests and minor fixes for mooncake connector ( #36946 )
...
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com >
2026-03-27 09:26:40 +01:00
Or Ozeri
7cc302dd87
[kv_offload+HMA][7/N]: Support register_kv_caches for hybrid models ( #37853 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-03-27 08:38:33 +03:00
Wei Zhao and root
0904b6550d
Fix multi-node allreduce fusion ( #38136 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: root <root@theia0053.lyris.clusters.nvidia.com >
2026-03-26 20:24:36 +00:00
Andreas Karatzas
bdc1719eb9
[ROCm][CI] Fix AITER state leak in shared_fused_moe_routed_transform test ( #38137 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-26 09:26:46 -07:00
yjz
b745e8b5d3
[KVTransfer][Mooncake] Add heterogeneous TP support for disaggregated P/D in MooncakeConnector ( #36869 )
...
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com >
2026-03-25 14:24:07 +01:00
Or Ozeri