 Kfir Toledo
|
b89443b8d9
|
[KVConnector]: Enable Cross-layers KV cache layout for MultiConnector (#30761)
Signed-off-by: Kfir Toledo <kfir.toledo@ibm.com>
|
2026-01-07 16:59:43 +00:00 |
|
|
BlankRandWentao Ye
|
0790f07695
|
[Misc] Improve error messages for unsupported types and parameters (#30593)
Signed-off-by: BlankR <hjyblanche@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-01-07 09:00:16 +00:00 |
|
|
weiyu
|
e7596371a4
|
[Refactor][TPU] Remove torch_xla path and use tpu-inference (#30808)
Signed-off-by: Wei-Yu Lin <weiyulin@google.com>
Signed-off-by: weiyu <62784299+weiyu0824@users.noreply.github.com>
|
2026-01-07 16:07:16 +08:00 |
|
|
tjp_zjuandCyrus Leung
|
55caa6051d
|
refactor: find_loaded_library (#31866)
Signed-off-by: tjp_zju <tanjianpingzju1990@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-01-07 06:42:20 +00:00 |
|
|
Yihua Cheng
|
5b833be49e
|
[1/2][lmcache connector] clean up lmcache multi-process adapter (#31838)
Signed-off-by: ApostaC <yihua98@uchicago.edu>
|
2026-01-07 02:02:42 +00:00 |
|
|
kzwrime
|
21156ff199
|
[Bugfix] Add missing extra_tensors arg to DeviceCommunicatorBase.disp… (#31644)
Signed-off-by: kunzh <zhikun.wu@outlook.com>
|
2026-01-06 01:26:09 +08:00 |
|
|
Yihua Cheng
|
b471aad41f
|
[KVconnector][LMCache] remove the import of legacy LMCache code (#31704)
Signed-off-by: ApostaC <yihua98@uchicago.edu>
|
2026-01-05 10:11:01 +00:00 |
|
|
Tyler Michael Smith
|
fe8a9fbd2e
|
[Bugfix] Fix EPLB state logging error (#31455)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-01-05 04:06:28 +00:00 |
|
|
Nick Hill
|
bd877162eb
|
[BugFix] Support online dense model DP without overhead (#30739)
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
|
2026-01-02 23:36:38 +08:00 |
|
|
SageandSage
|
39512aba72
|
[Prefix Cache] Include lora_name in BlockStored event for deterministic KV-cache reconstruction (#27577)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Co-authored-by: Sage <80211083+sagiahrac@users.noreply.github.com>
|
2025-12-30 00:17:16 +00:00 |
|
|
qli88
|
0f35429a0c
|
[CI]Test Group 'NixlConnector PD accuracy tests' is fixed (#31460)
Signed-off-by: qli88 <qiang.li2@amd.com>
|
2025-12-29 23:48:56 +00:00 |
|
|
chunxiaozheng
|
b12cb38398
|
implements register kv caches in lmcache connector (#31397)
Signed-off-by: idellzheng <idellzheng@tencent.com>
|
2025-12-29 11:13:42 -08:00 |
|
|
Cyrus Leung
|
aa3868ecfe
|
[Chore] Remove unused noqas (#31263)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-24 05:38:46 -08:00 |
|
|
Chao Lei
|
8b59753cdb
|
[P/D] Mooncake connector support more protocols (#30133)
Signed-off-by: LCAIZJ <leichao139636@163.com>
|
2025-12-24 10:24:07 +08:00 |
|
|
Chendi.XueandNicolò Lucchesi
|
6ca74bc11a
|
[NIXL][BUG FIX] Fix both failing issue and accuracy issue with nixl + host_buffer on CUDA (#30419)
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
|
2025-12-18 22:10:02 +00:00 |
|
|
wz1qqxandwz1qqx
|
889f8bb250
|
[BugFix]Reclaim resources to prevent memory leaks when use LMCacheMPConnector (#30745)
Signed-off-by: wz1qqx <ziqi.wang@novita.ai>
Co-authored-by: wz1qqx <ziqi.wang@novita.ai>
|
2025-12-18 19:09:51 +00:00 |
|
|
wangxiyuan
|
a85724bd6e
|
[Platform] Let EPD work with non-cuda platform (#30225)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
|
2025-12-18 06:45:29 +00:00 |
|
|
Yihua Cheng
|
ec965569d9
|
[KV connector][LMCache] Only record the cuda event when there are request to store/load (#30814)
Signed-off-by: ApostaC <yihua98@uchicago.edu>
|
2025-12-18 05:31:34 +00:00 |
|
|
Nicolò Lucchesi
|
bc3700e0cd
|
[NIXL] Support P tensor-parallel-size > D tensor-parallel-size (#27274)
Signed-off-by: NickLucche <nlucches@redhat.com>
|
2025-12-18 11:53:30 +08:00 |
|
|
Roger WangandSun Kim
|
f5f51e5931
|
[Core][MM] Optimize encoder cache manager by operating with embeddings only (#30475)
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Sun Kim <sunytokki@gmail.com>
|
2025-12-16 14:18:17 -08:00 |
|
|
jiahanc
|
254a7f8fd6
|
[Perf] Do FP4 quant before All gather on flashinfer trtllmgen MOE (#30014)
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
|
2025-12-16 13:01:48 -08:00 |
|
|
Chendi.XueandMark McLoughlin
|
ae2e503dda
|
[NIXL][BUG FIX] Fix a bug for PD with host_buffer after merging 29665 (#30420)
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
|
2025-12-14 15:38:28 +00:00 |
|
|
|
97f2f160fd
|
[ROCm][CI] Add "Qwen3-Next-80B-A3B-Instruct MTP Async EPLB Accuracy Test" Back Into AMD CI (#30590)
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-12-14 06:56:26 +00:00 |
|
|
Qier LiandQier Li
|
dc7fb5bebe
|
[Bug][KVConnector][Metrics] Remove a vacuous assertion breaking external-launcher (#30577)
Co-authored-by: Qier Li <qier@fb.com>
|
2025-12-14 01:23:08 +00:00 |
|
|
Nick Hill
|
947dfda9c2
|
[LMCache] Relax lmcache version requirement (#30425)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-12-11 18:18:47 -09:00 |
|
|
Martin Hickey
|
f4417f8449
|
[KVConnector] Add KV events to KV Connectors (#28309)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2025-12-11 15:30:29 +01:00 |
|
|
Sage Moore
|
b4054c8ab4
|
Revert "[CI] Add Async Eplb nightly CI tests (#29385)" (#30431)
|
2025-12-11 00:48:35 +00:00 |
|
|
Christina Norman
|
166ac3c94d
|
fix(shm): Add memory barriers for cross-process shared memory visibility (#30407)
Signed-off-by: Christina Holland <hey@christinaholland.com>
Signed-off-by: Christina <truffle@gmail.com>
|
2025-12-10 23:01:19 +00:00 |
|
|
Nick Hill
|
6ccb7baeb1
|
[LMCache] Fix breakage due to new LMCache version (#30216)
Signed-off-by: Nick Hill <nhill@redhat.com>
|
2025-12-10 11:52:01 -08:00 |
|
|
Jialin Ouyang
|
9f042ba26b
|
[Perf] Enable environment cache in EngineCore to enable the feature for UniProcExecutor as well (#29289)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
2025-12-10 14:13:01 -05:00 |
|
|
Or Ozeri
|
4c6fd25880
|
kv_transfer: Rename the shared storage connectors (#30201)
Signed-off-by: Or Ozeri <oro@il.ibm.com>
|
2025-12-08 20:46:09 -08:00 |
|
|
Lain
|
1fb632fdb6
|
[Perf] Improve fp8 quant in mla; replace ReduceSum with ReduceScatterSum (#29795)
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
|
2025-12-08 15:02:34 -08:00 |
|
|
weiguihua2
|
398a596ed2
|
[MP executor] fix get device count for multi node of mp executor feature (#30042)
Signed-off-by: weiguihua2 <weiguihua2@huawei.com>
|
2025-12-09 01:33:48 +08:00 |
|
|
AuruTus
|
8d3da4c79d
|
[MISC]: change NIXL compatibility hash logging level to debug (#30182)
|
2025-12-07 00:21:03 +00:00 |
|
|
kxand01267596
|
d6aeaddf4a
|
[bugfix] fix type[AttentionBackend] bug in kv_connector_base_v1 (#30051)
Signed-off-by: 01267596 <xiongkai123@cmbchina.com>
Co-authored-by: 01267596 <xiongkai123@cmbchina.com>
|
2025-12-06 07:11:31 +00:00 |
|
|
Tova Movshovitzandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
adb315060c
|
[KVConnector][Feature] Support KV connector cache reset via /reset_prefix_cache (#27170)
Signed-off-by: tovam <tovam@pliops.com>
Signed-off-by: Tova Movshovitz <tovam@pliops.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-12-05 18:33:26 +00:00 |
|
|
Mark McLoughlin
|
dff0a2b394
|
[NIXL] Add remote_request_id to kv_transfer_params (#29665)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
|
2025-12-05 09:43:48 -08:00 |
|
|
Nicolò LucchesiandCyrus Leung
|
78c44fd722
|
[NIXL] Small cleanup of unused variables (#29618)
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-12-05 18:17:36 +01:00 |
|
|
Mark McLoughlin
|
949a6a19d2
|
[NIXL] Add compatibility checking to NIXL KV connector handshake (#29503)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
|
2025-12-05 15:52:45 +01:00 |
|
|
|
1119f6e47a
|
Abstract eplb algo (#26471)
Signed-off-by: Che Ruan <cr623@ic.ac.uk>
Signed-off-by: mengxingkongzhouhan <117415539+mengxingkongzhouhan@users.noreply.github.com>
Signed-off-by: Mercykid-bash <ruanche0218@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Che Ruan <cr623@ic.ac.uk>
Co-authored-by: mengxingkongzhouhan <117415539+mengxingkongzhouhan@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2025-12-04 19:09:09 +00:00 |
|
|
Kuntai Du
|
ece2825a29
|
[KVConnector] Remove v0-related kv connector components such as kv pipe and kv lookup buffer (#29705)
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
|
2025-12-04 18:20:48 +00:00 |
|
|
dtcandNicolò Lucchesi
|
842aba501d
|
[P/D] Introduce Mooncake Transfer Engine as kv_connector (#24718)
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
Signed-off-by: dtc <dtcccc@linux.alibaba.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
|
2025-12-04 09:51:36 +00:00 |
|
|
Kuntai Du
|
8aaa81b35f
|
[KVConnector] remove unused code (the model aware kv ops class) (#29709)
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
|
2025-12-04 06:00:52 +00:00 |
|
|
Xieyang Xu
|
ad32e3e19c
|
enable multi-node in external launcher mode (#29833)
|
2025-12-03 17:02:02 -08:00 |
|
|
WeiQing ChenandCyrus Leung
|
7fe9c1a223
|
[CI] Add Async Eplb nightly CI tests (#29385)
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-12-03 09:51:08 +00:00 |
|
|
|
fc6acc88ca
|
[Bugfix] Missing cached item in the MultiModalReceiverCache (#28525)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Chenguang Zheng <645327136@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2025-12-01 10:18:07 -08:00 |
|
|
Didier Durand
|
04a797cd0e
|
[Doc]: fixing typos in various files. (#29717)
Signed-off-by: Didier Durand <durand.didier@gmail.com>
|
2025-11-29 01:15:39 -08:00 |
|
|
Cyrus Leung
|
9e6bcda3ac
|
[mypy] Enable type checking for more directories (#29674)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-11-28 08:39:27 -08:00 |
|
|
Cyrus Leung
|
0808eb813b
|
[Misc] Remove yapf directives (#29675)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-11-28 15:07:23 +00:00 |
|
|
Cyrus Leung
|
a24ea5414b
|
[Deprecation] Advance deprecation status (#29617)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-11-27 19:04:58 +00:00 |
|