iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15,702 Commits 3 Branches 0 Tags
d4cb783c10ffc091af7f09a3b052dceadc06d075
Commit Graph
311 Commits
Author SHA1 Message Date
Ronen Schaffer a5b17fba8f [KV Offload] Implement shutdown() in OffloadingConnector and related classes (#39182) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-04-10 08:06:22 +03:00
Yuwei Anandyuwei 467d3247c3 [LMCache] vLLM Block Allocation Event (#38856) 
Signed-off-by: yuwei <yuwei@dev.local>
Co-authored-by: yuwei <yuwei@dev.local>
 2026-04-09 17:30:29 -07:00
Chendi.Xue ef5a226819 [PD][HeteroArch]Fix accuracy issue with CPU_ATTN as Decoder and Flash_ATTN as prefiller (#38935) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
 2026-04-09 11:19:07 +08:00
Or Ozeri 512c5eb455 [kv_offload+HMA][5/N]: Track group block hashes and block IDs (#37109) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-04-08 19:50:28 +03:00
ibifrostandSimon Mo 96b5004b71 [KVConnector] Support 3FS KVConnector (#37636) 
Signed-off-by: wuchenxin <wuchenxin.wcx@alibaba-inc.com>
Signed-off-by: ibifrost <47308427+ibifrost@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2026-04-07 15:46:00 +00:00
maobaolongandChauncey 729eb59f60 [KVConnector]: prioritize external connector over internal registry (#38301) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-04-07 15:03:11 +00:00
zhanqiuhu bfdc0a3a99 [NIXL][Mamba][3/N] Heterogeneous TP: 3-read conv state transfer (#37635) 2026-04-06 19:07:02 +02:00
yzong-rhandNicolò Lucchesi dc0428ebb8 [NIXL][BUG] Fix Triton heterogeneous TP (#37940) 
Signed-off-by: Yifan <yzong@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-04-01 17:23:15 +02:00
yjzandNicolò Lucchesi 992368522f [KVTransfer] Fix TpKVTopology.is_kv_replicated equality case (#38179) 
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-04-01 12:41:49 +02:00
Yifan Qiao 91e4521f9f [Feat][v1] Simple yet General CPU KV Cache Offloading (#37160) 
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
 2026-03-31 17:58:37 -07:00
Kfir ToledoandOr Ozeri 6cc7abdc66 [kv_offload+HMA] Fix num_blocks with different per-layer page sizes and improve assert message (#38554) 
Signed-off-by: Kfir Toledo <kfir.toledo@ibm.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-03-31 06:00:40 +00:00
dtc 6287e7fa20 [P/D] Mooncake: Add unit tests and minor fixes for mooncake connector (#36946) 
Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
 2026-03-27 09:26:40 +01:00
Or Ozeri 7cc302dd87 [kv_offload+HMA][7/N]: Support register_kv_caches for hybrid models (#37853) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-27 08:38:33 +03:00
yjz b745e8b5d3 [KVTransfer][Mooncake] Add heterogeneous TP support for disaggregated P/D in MooncakeConnector (#36869) 
Signed-off-by: JianDan0212 <zhangyj0212@gmail.com>
 2026-03-25 14:24:07 +01:00
Wentao Ye c59a132f96 [V0 Deprecation] Refactor kv cache from list to element (#37487) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-23 20:10:11 -07:00
Wentao Ye b3d0b37908 [Refactor] Remove unused dead code (#36171) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-20 16:12:51 -06:00
Martin Hickey 880be2b1b8 [Metrics] Some small refactoring for better maintainability (#33898) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-03-20 16:11:34 +00:00
zhanqiuhu d49f273144 [SSM/Mamba] Follow-up: N-1 prefill for P/D disaggregation (#37310) 2026-03-19 08:22:00 +01:00
Wentao Ye 0d81a1fe61 [V0 Deprecation] Deprecate virtual engine (#37195) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-18 14:30:14 -07:00
Or Ozeri 5dd8df0701 [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-18 19:26:40 +02:00
Or Ozeri 525f2eeb0b [kv_offload+HMA][6/N]: Split offloading_connector.py (#37405) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-18 14:42:46 +01:00
Andy Lo 98b09ddc27 [NIXL][Bugfix] metrics & testing minor bug (#36051) 
Signed-off-by: Andy Lo <andy@mistral.ai>
 2026-03-18 14:39:14 +01:00
Or OzeriandNicolò Lucchesi fcf0687b27 [kv_offload+HMA][0/N]: Support block-level preemption handling (#34805) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-18 08:49:53 +02:00
youkaichao 0fb142a454 [perf][connector] optimize build_connector_meta when host buffer transfer is not used (#37165) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2026-03-17 11:59:35 +00:00
zhanqiuhu 2dccb38f73 [Bugfix][MultiConnector] Fix MultiConnector for SupportsHMA sub-connectors (#36549) 2026-03-16 20:51:04 +00:00
Nicolò Lucchesi f5c081d432 [PD][Nixl] Add support for hybrid SSM-FA models (#36687) 2026-03-16 19:58:06 +01:00
haosdentandOr Ozeri ca1954d58c [Bugfix] Disable cross-layer KV cache for MLA attention backends (#37090) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
 2026-03-16 19:03:10 +02:00
Fynn Schmitt-Ulms 04bf5a35fa [Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013) 2026-03-16 14:53:45 +01:00
rasmith 0024f39a32 [ROCm][P/D][MORI][BugFix] Add transfer_id for moriio_connector so moriio_connector to restore P/D functionality (#34907) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2026-03-16 10:36:51 +08:00
Harry Mellor 5a3f1eb62f [Misc] Set default kv_buffer_device in a better way (#36862) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-13 19:07:33 +00:00
Or Ozeri cfaf4668f7 [kv_offload+HMA][1/N]: Support multiple KV groups in OffloadingSpec (#36610) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2026-03-13 08:04:21 +00:00
Kunshang Ji 53ec16a705 [Hardware] Replace torch.cuda.device_count/current_device/set_device API (#36145) 
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-12 07:57:47 -07:00
sfeiqiangandphaedonsun 8cb24d3aed [KV Connector] Support using FlexKV as KV Cache Offloading option. (#34328) 
Signed-off-by: phaedonsun <phaedonsun@tencent.com>
Co-authored-by: phaedonsun <phaedonsun@tencent.com>
 2026-03-12 00:46:20 -07:00
Yan Ma 894843eb25 replace with torch.cuda.device with with torch.accelerator.device_index (#36144) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2026-03-11 23:12:57 -07:00
Yuwei An 2ef69456f5 [LMCache] Fault Tolerance Mechanism (#36586) 
Signed-off-by: Oasis-Git <ayw.sirius19@gmail.com>
 2026-03-12 03:54:39 +00:00
maobaolongandYihua Cheng 12001f2ebc [LMCache] Pass TP size in lookup for MLA multi-reader locking (#36129) 
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Co-authored-by: Yihua Cheng <yihua98@uchicago.edu>
 2026-03-11 20:45:20 +00:00
Or OzeriandNicolò Lucchesi 7ee5d5093b [BugFix][kv_offload] Fix offloading decodes with async scheduling (#33881) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-11 20:43:40 +00:00
Wentao Ye 35bdca5431 [Refactor] Remove dead code in KV connector (#36424) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-11 19:40:17 +00:00
Or OzeriandNicolò Lucchesi a1a3523a56 [KVConnector] Support worker -> scheduler metadata (#31964) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-03-11 17:36:37 +00:00
Nicolò Lucchesi 098d844731 [NIXL][1/N] Refactor kernel_block_size detection (#35752) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-11 01:11:23 -07:00
liuzhenweiandKunshang Ji f22d6e0267 [Hardware][NIXL] set default kv buffer type for different platform (#36438) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-11 05:19:28 +00:00
Harry Mellor f83b933b84 [CI] Bump mypy version to 1.19.1 (#36104) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-10 09:18:28 -07:00
youkaichao f85b4eda3a [bugfix] fix nvlink for nixl/ucx (#36475) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2026-03-10 07:49:47 +08:00
Matthew Bonanni 77a73458e3 Reapply [Attention] Refactor check_and_update_config (#35122) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-03-09 07:17:14 -07:00
Samuel Shen 5d6aae4577 [LMCache MP Patch]: Race Condition + Duplicated Block Ids (#35831) 2026-03-07 13:52:48 -08:00
Roy Huang 63298ee173 [Bugfix][LMCache][KVConnector] fix potential memory leak in LMCache multiprocess mode (#35931) 2026-03-07 13:52:35 -08:00
Nicolò Lucchesi 5b3ba94ab4 [Core][KVConnector] Support HMA+NixlConnector (#35758) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-03-06 08:51:21 +01:00
Jiayi YanandHarry Mellor 6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Harry Mellor 17dc9c7fc9 [CI] Bump mypy version (#34950) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 20:55:11 +00:00
liuzhenweiandKunshang Ji 9dd656f0ea [XPU][NIXL] Add GPUDirect RDMA support for XPU (#35270) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-03 08:42:49 +08:00