iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
7,797 Commits 3 Branches 0 Tags
4adc66f64d56338489d00d94de6e13d95741c4be
Commit Graph
259 Commits
Author SHA1 Message Date
Woosuk Kwon 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Zhonghua Deng 8a4e5c5f3c [V1][P/D]Enhance Performance and code readability for P2pNcclConnector (#20906) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
 2025-07-16 22:13:00 -07:00
Trevor Morrisandmgoin a8593237c0 Add pynccl all-gatherv and reducescatterv (#20154) 
Signed-off-by: Trevor Morris <tmorris@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 18:59:23 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 53fa457391 [Misc] Add unit tests for MoE ModularKernel combinations + Profiling utility (#20449) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-11 07:51:46 -07:00
Nick HillandDavid Ben-David 574ad60db9 [KVConnector] Always call connector clear_metadata() at end of step (#20756) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: David Ben-David <sdavidbd@gmail.com>
 2025-07-10 22:37:27 +01:00
Or Ozeri cc876d0f29 [KVConnector] Aggregate finished requests on the scheduler (#19555) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-07-10 09:22:18 +01:00
YimingleiyimingNick Hill
 		cd587c93ef [BugFix]: Properly set engine_id when using multi connector (#19487) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: leiyiming <leiyiming@kingsoft.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-07-09 20:32:44 +00:00
Liangliang Maandzhenwei-intel a3e4e85ece [XPU][CI] enhance xpu test support (#20652) 
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
Co-authored-by: zhenwei-intel <zhenweiliu@habana.ai>
 2025-07-09 16:53:09 +00:00
Nicolò Lucchesi 71d1d75b7a [PD][Nixl] Remote consumer READ timeout for clearing request blocks (#20139) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-08 08:56:40 +01:00
Jee Jee Li 1caca5a589 [Misc] Add SPDX-FileCopyrightText (#20428) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-04 07:40:42 +00:00
Nicolò Lucchesi 8d775dd30a [Misc] Fix Unable to detect current VLLM config. Defaulting to NHD kv cache layout warning (#20400) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-03 14:56:09 -07:00
Ning Xie 1dba2c4ebe [Misc] adjust for ipv6 for mookcacke url parse (#20107) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-07-03 20:27:17 +00:00
Woosuk Kwon 7f280d69c9 [Optimization] Cache sampled token ids in model runner (#20291) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-01 11:01:31 -07:00
Nicolò Lucchesi 650d5dbd04 [Misc] Minor refactor of NIXL background handshake (#20068) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-07-01 12:40:14 +01:00
Michael Goin be250bbc67 [V1] Only print cudagraph tqdm on rank 0 with is_global_first_rank (#19516) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-01 06:02:09 +00:00
Zhonghua Deng ded1fb635b [Bugfix][V1][P/D]Fix the issue of occasional garbled output for P2pNcclConnector (#20263) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
 2025-06-30 16:45:14 -07:00
Woosuk Kwon 2863befce3 [Optimization] Use Shared CachedRequestData Instance Across All Requests (#20232) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 09:07:50 -07:00
Wentao Ye 4d36693687 [Refactor] Create a function util and cache the results for has_deepgemm, has_deepep, has_pplx (#20187) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-28 22:06:38 +00:00
li haoyangandilmarkov 0740e29b66 [Feature] add quick all reduce (#19744) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-06-26 20:54:24 -07:00
Bowen Wang e9fd658a73 [Feature] Expert Parallelism Load Balancer (EPLB) (#18343) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
 2025-06-26 15:30:21 -07:00
Nicolò Lucchesi 2582683566 [PD] Skip tp_size exchange with rank0 (#19413) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-25 20:04:39 -07:00
Nick Hill 55c65ab495 [P/D] Avoid stranding blocks in P when aborted in D's waiting queue (#19223) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-25 15:19:44 -07:00
Nick Hill c40692bf9a [Misc] Add parallel state node_count function (#20045) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-25 13:38:53 -07:00
lkchenandNick Hill 91f7d9d0b6 [P/D] Asynchronously do _nixl_handshake (#19836) 
Signed-off-by: Linkun Chen <github@lkchen.net>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-24 12:46:10 -07:00
lkchen d0132f025d [Misc] Add type alias ReqId and EngineId for better readability (#19880) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-06-23 12:57:57 -07:00
lkchen 1bcd15edc7 [BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when *all* transfer done (#19874) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-06-22 22:41:53 -07:00
Nicolò LucchesiandNick Hill 2ebff5b77c [P/D][NixlConnector] Support tp_size > num_kv_heads deployments (#19691) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-22 22:41:50 -07:00
Zhonghua Deng eccdc8318c [V1][P/D] An native implementation of xPyD based on P2P NCCL (#18242) 
Signed-off-by: Abatom <abzhonghua@gmail.com>
 2025-06-18 06:32:36 +00:00
Jiayi Yao cda92307c1 [Misc] Update lmcache connector with the latest connector apis (#19441) 
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
 2025-06-17 19:57:54 +00:00
Nicolò Lucchesi 4c8f64faa7 [V1][Kernel] Flashinfer HND KV cache layout (#19280) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-17 09:09:22 -04:00
Conroy Cheers 0860087aff [Fix] Fall back to Gloo when NCCL backend is unavailable (#19641) 
Signed-off-by: conroy-cheers <conroy@corncheese.org>
 2025-06-17 08:42:14 +08:00
Nicolò Lucchesi 1129e2b1ab [V1][NixlConnector] Drop num_blocks check (#19532) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-12 12:36:14 +00:00
Varun Sundar RabindranathandVarun 5cf2daea9a [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-09 10:50:39 -04:00
Nick Hill aad30bd306 [BugFix] Fix MultiConnector test after HMA changes (#19291) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 20:16:24 +00:00
Nick Hill 65c69444b1 [Docs] Improve V1 KVConnector interface documentation (#19172) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 16:22:45 +08:00
Nicolò Lucchesi 9ef9173cfa [P/D][NixlConnector] Enable FlashInfer backend (#19090) 2025-06-05 17:10:15 +00:00
Povilas Kanapickas 85e2b7bb13 [MISC][Bugfix] Use less CPU when message queue has been empty for some time (#16226) 
Signed-off-by: Povilas Kanapickas <povilas@radix.lt>
 2025-06-05 16:53:08 +00:00
Robert ShawandNick Hill c56ed8bb0e [Bugfix][Nixl] Fix full prefix cache hit bug (#18632) 
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-05 02:07:32 +00:00
Nicolò Lucchesi b2fac67130 [P/D] Heterogeneous TP (#18833) 
Signed-off-by: nicklucche <nlucches@redhat.com>
 2025-06-04 23:25:34 +00:00
Tyler Michael Smith d459fae0a2 [Bugfix][EP+DP] Fix internode check (#19112) 
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
 2025-06-04 23:39:23 +08:00
Yan Ru Pei b712be98c7 feat: add data parallel rank to KVEventBatch (#18925) 2025-06-03 17:14:20 -07:00
Nicolò LucchesiandRobert Shaw 5d96533e22 [Bugfix][P/D] Fix Prefix Cache Bug (#18411) 
Signed-off-by: nicklucche <nlucches@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-06-03 23:53:16 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath fa98d77773 [Kernel] DeepEP dispatch-combine kernel integration (#18434) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-03 12:30:02 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Li, Jiang 8655f47f37 [CPU][CI] Re-enable the CPU CI tests (#19046) 
Signed-off-by: jiang.li <jiang1.li@intel.com>
 2025-06-02 20:46:47 -07:00
Tyler Michael Smith 8a57872b2a [Bugfix][EP+DP] Use pplx-kernel internode instead of intranode (#19034) 
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-06-03 11:36:51 +08:00
Siyuan LiuHossein SarsharChengji Yao
 		9112b443a0 [Hardware][TPU] Initial support of model parallelism with single worker using SPMD (#18011) 
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-06-03 00:06:20 +00:00
Robert Shaw b9f61e1387 [Bugfix][Nixl] Fix DP Metadata Handshake (#19008) 
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
 2025-06-02 03:30:41 +00:00
rongfu.leng aa54a7bf7b [BugFix] fix data parallel construct ipv6 url addres (#18991) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-06-01 11:42:10 +08:00
ptarasiewiczNV 8bf507d766 [P/D] NixlConnector use cache device index for memory registration (#18969) 
Signed-off-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>
 2025-05-31 11:19:18 -04:00