iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1000 Commits
Author SHA1 Message Date
fhl2000 2e8cbb58f3 [BugFix] Fix full cuda graph slot_mapping (#21228) 
Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
 2025-07-19 14:13:18 -07:00
Thomas Parnell 881e3cbe3b [V1] [Hybrid] Enable piecewise CUDA Graph for mamba layers (#21194) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-19 19:27:21 +00:00
kourosh hakhamaneshi 9f414a12ad [BugFix] Make PD work with Ray (#21072) 
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
 2025-07-19 08:46:50 -07:00
Li, Jiang e3a0e43d7f [bugfix] Fix auto thread-binding when world_size > 1 in CPU backend and refactor code (#21032) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-19 05:13:55 -07:00
Lucas Wilkinson 59f935300c [BugFix] Fix potential cuda-graph IMA (#21196) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-19 02:18:47 -07:00
Lucas Wilkinson 468e2400fe [BugFix][CPU] Fix TorchSDPABackendImpl doesn't have use_irope (#21200) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-18 23:18:48 -07:00
Lucia FangandLu Fang 9a9fda1423 [Core] Support Local Chunked Attention for Hybrid KV Cache (#19351) 
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lu Fang <fanglu@meta.com>
 2025-07-18 20:48:38 -07:00
Rui Qiao 217937221b Elastic Expert Parallel Initial Support (#20775) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-18 17:46:09 -07:00
Cyrus Leung 45badd05d0 [Core] Set pooling params based on task and model (#21128) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-18 05:41:17 -07:00
Roger Wangandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> ba2dfbb0c2 [Misc] Make MM embedding merge interface explicit in model runner (#21147) 
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-07-18 07:13:57 +00:00
Lucas Wilkinson 89cab4d01f [Attention] Make local attention backend agnostic (#21093) 2025-07-18 00:10:42 -04:00
Eric Curtin ac9fb732a5 On environments where numa cannot be detected we get 0 (#21115) 
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
 2025-07-17 18:52:17 +00:00
Lucas Wilkinson 76b494444f [Attention] Refactor attention metadata builder interface (#20466) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-17 04:44:25 +00:00
Chengji Yao 85431bd9ad [TPU] fix kv_cache_update kernel block size choosing logic (#21007) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-07-16 04:39:48 +00:00
Peter Pan 1eb2b9c102 [CI] update typos config for CI pre-commit and fix some spells (#20919) 
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
 2025-07-15 21:12:40 -07:00
Thomas Parnell 3534c39a20 [V1] [Hybrid] Refactor mamba state shape calculation; enable V1 via cli (#20840) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-15 04:04:35 -07:00
Yifei Teng c586b55667 [TPU] Optimize kv cache update kernel (#20415) 
Signed-off-by: Yifei Teng <tengyifei88@gmail.com>
 2025-07-15 03:56:43 -07:00
wangxiyuan 1e9438e0b0 [MISC] Move bind_kv_cache to worker module (#20900) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-07-14 09:40:00 +00:00
22quinn 8632e831ba [Core] Add update_config RPC method (#20095) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-07-14 00:49:18 +00:00
lkchen f56d2996ca [Misc] Respect no_use_tqdm_on_load flag while capturing CUDA graph (#20834) 
Signed-off-by: Linkun <github@lkchen.net>
 2025-07-11 23:04:45 -07:00
nopperl 5d09152ff1 [V1] Enable Mamba2 layers other than MambaMixer2 in the v1 engine (#20660) 
Signed-off-by: nopperl <54780682+nopperl@users.noreply.github.com>
 2025-07-11 05:53:31 +00:00
Nick HillandDavid Ben-David 574ad60db9 [KVConnector] Always call connector clear_metadata() at end of step (#20756) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: David Ben-David <sdavidbd@gmail.com>
 2025-07-10 22:37:27 +01:00
Or Ozeri cc876d0f29 [KVConnector] Aggregate finished requests on the scheduler (#19555) 
Signed-off-by: Or Ozeri <oro@il.ibm.com>
 2025-07-10 09:22:18 +01:00
Chenyaaang fdfd409f8f [TPU][Core]Make load weight exceed hbm error more instructive for customers (#20644) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-07-10 07:01:17 +00:00
Nick Hill 59389c927b [BugFix][CPU] Fix CPU worker dependency on cumem_allocator (#20696) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-10 14:24:20 +08:00
Liangliang Maandzhenwei-intel a3e4e85ece [XPU][CI] enhance xpu test support (#20652) 
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
Co-authored-by: zhenwei-intel <zhenweiliu@habana.ai>
 2025-07-09 16:53:09 +00:00
B-201 6bbf1795b7 [Misc] Fix the size of batched_dummy_mm_inputs in profile_run (#20434) 
Signed-off-by: bk-201 <joy25810@foxmail.com>
 2025-07-08 20:15:44 -07:00
Akash kaothalkarAkash KaothalkarNikhil Gupta
 		6db31e7a27 [Hardware][PPC64LE] Enable V1 for ppc64le and ARM (#20554) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Nikhil Gupta <nikhil.gupta2@arm.com>
 2025-07-08 20:00:41 -07:00
Ratnam Parikh c40784c794 [BugFix][Intel GPU] Use refactored API for dist_backend in V1 worker (#20596) 
Signed-off-by: ratnampa <ratnam.parikh@intel.com>
 2025-07-08 19:44:23 -07:00
Kunshang Ji 0b407479ef [misc]refactor Platform.set_device method (#20262) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-09 01:39:47 +00:00
Sanger SteelandEta 72d14d0eed [Frontend] [Core] Integrate Tensorizer in to S3 loading machinery, allow passing arbitrary arguments during save/load (#19619) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Eta <esyra@coreweave.com>
 2025-07-07 22:47:43 -07:00
Yang YangandKunshang Ji 6e2c19ce22 [Refactor]Abstract Platform Interface for Distributed Backend and Add xccl Support for Intel XPU (#19410) 
Signed-off-by: dbyoung18 <yang5.yang@intel.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-07-07 04:32:32 +00:00
Woosuk Kwon 9528e3a05e [BugFix][Spec Decode] Fix spec token ids in model runner (#20530) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-06 19:44:52 +00:00
Thomas ParnellStanislaw WozniakTyler Michael SmithChen Zhang
 		2f35a022e6 Enable V1 for Hybrid SSM/Attention Models (#20016) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
 2025-07-04 17:46:53 +00:00
Jee Jee Li 1caca5a589 [Misc] Add SPDX-FileCopyrightText (#20428) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-04 07:40:42 +00:00
afeldman-nmandNick Hill 48fb076cbc [V1] LogitsProcessor programming model (#16728) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-07-02 09:10:42 -07:00
Chengji Yao 7da296be04 [TPU] kv cache update kernel supports dynamic grid (#20235) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-07-02 06:33:37 +00:00
Woosuk Kwon 7f280d69c9 [Optimization] Cache sampled token ids in model runner (#20291) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-01 11:01:31 -07:00
Michael Goin be250bbc67 [V1] Only print cudagraph tqdm on rank 0 with is_global_first_rank (#19516) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-01 06:02:09 +00:00
Woosuk Kwon 2863befce3 [Optimization] Use Shared CachedRequestData Instance Across All Requests (#20232) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 09:07:50 -07:00
Woosuk Kwon 2062c0723d [Spec Decode] Refactor spec decoding into a separate function (#20238) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 08:13:50 -07:00
Woosuk Kwon 19108ef311 [Misc] Fix import (#20233) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-29 20:34:54 -07:00
Bowen Wang e9fd658a73 [Feature] Expert Parallelism Load Balancer (EPLB) (#18343) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
 2025-06-26 15:30:21 -07:00
Chengji Yao 04e1642e32 [TPU] add kv cache update kernel (#19928) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-26 10:01:37 -07:00
Kunshang Ji b69781f107 [Hardware][Intel GPU] Add v1 Intel GPU support with Flash attention backend. (#19560) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2025-06-26 09:27:18 -07:00
Chenyaaang 2d7620c3eb [TPU] Add TPU specific var VLLM_TPU_MOST_MODEL_LEN (#19919) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-06-25 15:51:02 -07:00
Chengji Yao 2cc2069970 [TPU][Bugfix] fix kv cache padding (#20048) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-25 21:24:10 +00:00
Li, Jiang 53da4cd397 [Bugfix][CPU] Fix InputBatch for pooling models in the CPU v1 (#20014) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-06-24 13:20:04 +00:00
Vadim Gimpelson 9a3b88328f [PERF] Speedup of MRoPE prepare inputs (#19939) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@centml.ai>
 2025-06-23 23:01:26 -07:00
Isotr0py 61f4fc5dc6 [Bugfix][v1] Fix step pooler implementation and step pooling usage in v1 (#19956) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-23 18:38:06 +00:00