iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
51 Commits
Author SHA1 Message Date
gnovack a532c83849 use 'max_active_experts' for moe lora input size (#33197) 
Signed-off-by: gnovack <gnovack@amazon.com>
 2026-02-27 03:50:43 +00:00
Chaojun ZhangandKunshang Ji 9f9a675b23 [XPU][8/N] Fix kernel bugs in XPU LoRA and MOE LORA (#34115) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-02-26 15:46:44 +08:00
Runkai Tao e1d97c38f8 [Bug Fix] Fix naive_block_assignment always defaulting to False due to arg misalignment (#33848) 
Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
 2026-02-12 11:30:57 +08:00
yugong333 ffe1fc7a28 Reduce the kernel overhead when num of active loras is smaller than max loras. Multiple cuda graphs are captured for each num of active-loras. (#32005) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
 2026-02-02 12:30:06 -05:00
Runkai Tao 7320ca3942 Add unpermute-aware fused MoE LoRA path (#32655) 
Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
 2026-02-02 09:46:09 +08:00
weiyu e7596371a4 [Refactor][TPU] Remove torch_xla path and use tpu-inference (#30808) 
Signed-off-by: Wei-Yu Lin <weiyulin@google.com>
Signed-off-by: weiyu <62784299+weiyu0824@users.noreply.github.com>
 2026-01-07 16:07:16 +08:00
ZT-AIAandJee Jee Li f84bf7d79b Add Loraconfig parameter to get_punica_wrapper function (#31408) 
Signed-off-by: ZT-AIA <1028681969@qq.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-12-29 22:27:31 -08:00
Didier Durand 66d3d5422c [Doc]: fixing typos in diverse files (#29492) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-11-27 07:15:50 -08:00
Jee Jee Li 1489902b53 [LoRA] Cleanup FusedMoEWithLoRA (#29187) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-22 04:01:30 +00:00
Jee Jee Li 9875be6431 [LoRA][2/2]Remove LoRA extra vocab (#28545) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-21 09:46:43 +08:00
gnovackandJee Jee Li d69062c67a add support for --fully-sharded-loras in fused_moe (#28761) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-19 16:32:00 +08:00
Xiaohong (Sean) ChenRobert ShawDanielle RobinsonHaipeng Lili2haipeng
 		d0c7792004 [Bugfix][LoRA][Spec Decode] Support LoRA with speculative decoding (#21068) 
Signed-off-by: Sean Chen <xiaohong_chen1991@hotmail.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Danielle Robinson <dcmaddix@gmail.com>
Co-authored-by: Haipeng Li <li2haipeng@gmail.com>
Co-authored-by: li2haipeng <44383182+li2haipeng@users.noreply.github.com>
 2025-11-08 01:58:22 +00:00
yugong333andJee Jee Li 2ec401bc39 Load tuned fused_moe_lora shrink and expand kernel configs separately (#27435) 
Signed-off-by: Yu Gong <yu3.gong@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-04 18:27:35 +08:00
gnovackandJee Jee Li 294c805f1d Early exit for MoE LoRA kernels (#27131) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-11-03 20:22:17 +08:00
Cyrus Leung 7c2bdb83dc [Misc] Clean up utils (#27552) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 09:05:40 +00:00
Danielle RobinsonDanielle RobinsonJee Jee Li
 		9932ed6a83 [Kernel] Adding split_K implementation for fused_moe_lora (#27291) 
Signed-off-by: Danielle Robinson <dmmaddix@amazon.com>
Signed-off-by: Danielle Robinson <dcmaddix@gmail.com>
Co-authored-by: Danielle Robinson <dmmaddix@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-27 02:05:24 -07:00
gnovackandJee Jee Li 8e4ca4d14e Bugfix - pass 'max_num_tokens_padded' into 'moe_lora_align_block_size' (#27311) 
Signed-off-by: gnovack <gnovack@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-22 12:23:57 +00:00
Chen WuwuchenNathan Van GheembanjuedeDanielle RobinsonJee Jee Libk-201
 		5f6cbf60d6 [Feature][Kernel]FusedMoE LoRA (#21229) 
Signed-off-by: wuchen <cntryroa@gmail.com>
Signed-off-by: banjuede <lmklhc@163.com>
Signed-off-by: Chen Wu <cntryroa@gmail.com>
Signed-off-by: Danielle Robinson <dmmaddix@amazon.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: wuchen <wuchen@zetyun.com>
Co-authored-by: Nathan Van Gheem <vangheem@gmail.com>
Co-authored-by: banjuede <lmklhc@163.com>
Co-authored-by: Danielle Robinson <dmmaddix@amazon.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
 2025-10-21 03:01:37 +00:00
Andy LoandJee Jee Li b63f2143f8 [LoRA] LoRA cuda graph specialization (#25914) 
Signed-off-by: Andy Lo <andy@mistral.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-20 04:21:09 +00:00
Cyrus Leung 4d4d6bad19 [Chore] Separate out vllm.utils.importlib (#27022) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-17 00:48:59 +00:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Ashwin PhadkeandJee Jee Li ab196edefb Remove LoRA bias support (#25807) 
Signed-off-by: Ashwin Phadke <ashwinphadke12@rediffmail.com>
Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-10 09:50:33 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Jee Jee Li 0fa673af4c [V0 deprecation] Clean up LoRA (#25686) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-25 18:12:33 +00:00
Nicolò LucchesiandYe Qi 4cf71cc88a [TPU] Deprecate xm.mark_step in favor of `torch_xla.sync (#25254) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-09-22 10:12:57 +00:00
Chaojun Zhang 862f2ef893 [XPU] Fix the bug of LoRA logits on the XPU platform (#24081) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2025-09-03 08:21:18 +08:00
Chaojun ZhangandJee Jee Li ec261b0291 [XPU] IPEX-optimized Punica Wrapper on XPU (#21703) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-28 16:43:37 +00:00
Jee Jee Li 1eaff27815 [V0 deprecation] Remove long context LoRA (#21169) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-19 02:15:41 -07:00
Woosuk Kwon 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Ning Xie 2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Akshat TripathiChengji YaoxihajunJorge de FreitasJorge de Freitas
 		643622ba46 [Hardware][TPU][V1] Multi-LoRA Optimisations for the V1 TPU backend (#15655) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Signed-off-by: xihajun <junfan@krai.ai>
Signed-off-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Signed-off-by: Jorge de Freitas <jorge@krai.ai>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: xihajun <junfan@krai.ai>
Co-authored-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Co-authored-by: Jorge de Freitas <jorge@krai.ai>
 2025-05-28 19:59:09 +00:00
Harry Mellor 9b5b39b650 Update deprecated type hinting in vllm/lora (#18128) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-14 03:57:59 -07:00
Akshat TripathiandChengji Yao c20ef40fd0 [Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
 2025-05-07 16:28:47 -04:00
Aaron Pham da4e7687b5 [Fix] Support passing args to logger (#17425) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-04-30 08:06:58 -07:00
Jee Jee Li 6db94571d7 [Misc] Remove LoRA log (#15388) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-24 20:43:48 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 8a8b30eac1 [Bugfix] LoRA V0 - Fix case where max_num_seqs is between cudagraph capture sizes (#15308) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-22 02:03:32 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 400d483e87 [Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-18 09:47:53 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 0b1cfa6180 [Kernel] LoRA - Enable CUDAGraphs for V1 (#14626) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-13 20:42:04 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 5ff0d32580 [V1] LoRA - Add triton kernels for V1 (#13096) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-03-10 17:27:53 -04:00
Jee Jee Li 5157338ed9 [Misc] Improve LoRA spelling (#13831) 2025-02-25 23:43:01 -08:00
Jee Jee Li 82cabf53a3 [Misc] Delete unused LoRA modules (#13151) 2025-02-12 08:58:24 -08:00
Sanju C Sudhakaran 2880e21e3d [Hardware][Intel-Gaudi] Enable long-contexts + LoRA support for Intel Gaudi (#12812) 
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
 2025-02-08 17:15:30 +08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Shanshan Shen a7d59688fb [Platform] Move get_punica_wrapper() function to Platform (#11516) 
Signed-off-by: Shanshan Shen <467638484@qq.com>
 2025-01-13 13:12:10 +00:00
Akshat TripathiOleg MosalovJee Jee LiIsotr0py
 		8bddb73512 [Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100) 
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-01-12 13:01:52 +00:00
Cyrus LeungandMaxime Fournioux d848800e88 [Misc] Move print_*_once from utils to logger (#11298) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
 2025-01-09 12:48:12 +08:00
Jee Jee LiandZhonghua Deng b278557935 [Kernel][LoRA]Punica prefill kernels fusion (#11234) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>
 2025-01-07 04:01:39 +00:00
Jee Jee Li 3cb5769883 [Misc] Minor improvements to the readability of PunicaWrapperBase (#11200) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-12-14 16:38:27 +00:00
Sanju C Sudhakaran 8195824206 [Hardware][Intel-Gaudi] Enable LoRA support for Intel Gaudi (HPU) (#10565) 
Signed-off-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
 2024-12-12 08:09:28 +00:00