iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
45 Commits
Author SHA1 Message Date
Michael Goin ff1f83b056 [Refactor] Replace activation: str with MoEActivation enum (#33843) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-02-11 17:29:32 -08:00
bnellnm d1481ba783 [MoE Refactor] Introduce MoERunner abstraction and move execution logic from FusedMoE to DefaultMoERunner (#32344) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-02-10 19:51:07 -05:00
杨朱 · KikiandClaude Opus 4.5 1a7894dbdf [Misc] Replace Optional[X] with X | None syntax (#33332) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-30 01:56:59 -08:00
bnellnm dc917cceb8 [MoE Refactor] Move select_experts from FusedMoEQuantMethod -> FusedMoE (#31996) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-22 18:21:35 -05:00
bnellnm 327a02d8db [MoE Refactor] Separate Router into OO Classes (#30623) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-18 11:40:49 -05:00
bnellnm e74698c27a [Misc][Refactor] Add FusedMoERouter object (#30519) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-01-08 20:52:55 +00:00
baonudesifeizhaiRobert ShawRobert Shaw
 		54c8924384 [MoE Refactor][5/N] Isolate zero expert to LongCatFlash (#28891) 
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: Dongjie Zou <85092850+baonudesifeizhai@users.noreply.github.com>
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2025-12-20 18:22:04 +00:00
Christina NormanIsotr0pyIsotr0pygemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		dc13c99eed fix(gguf): Disable bfloat16 for GGUF on blackwell device (#30408) 
Signed-off-by: Christina <truffle@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Christina Norman <christina@example.com>
Co-authored-by: Isotr0py <isotr0py@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-12 23:10:12 +08:00
bnellnmRobert ShawLucas WilkinsonTyler Michael Smith
 		00e5cbb967 [MoE][Refactor] Remove most arguments to FusedMoEMethodBase.apply (#29066) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-12-09 13:48:25 -08:00
Tsukasa OIandIsotr0py 58d5b3f514 [Model][Quantization] Restore MoE + GGUF models support (incl. Qwen3 MoE) by allowing Sideload Parameters (#30116) 
Signed-off-by: Tsukasa OI <floss_llm@irq.a4lg.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-09 05:30:05 +00:00
bnellnm 8f066146c3 [MoE][Refactor] Make select_experts a non-static method (#29067) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-11-24 13:38:04 -05:00
Luciano MartinsLuciano MartinsIsotr0py
 		c2612371ad [Model] Add Gemma3 GGUF multimodal support (#27772) 
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-18 08:56:29 -08:00
bnellnm 938772af03 [Kernels] Isolate modular kernel code from FusedMoEMethodBase subclasses. (#27123) 2025-11-04 21:59:45 +08:00
Isotr0py 6ac5e06f7c [Chore] Clean up pytorch helper functions in vllm.utils (#26908) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: isotr0py <2037008807@qq.com>
 2025-10-18 09:48:22 -07:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
XuruiYangandyangxurui 845adb3ec6 [Model] Add LongCat-Flash (#23991) 
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
 2025-09-24 21:53:40 -07:00
Michael Goin 7361ab379f Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 22:48:40 +00:00
bnellnm 5963b98b46 [Kernel] Delegate construction of FusedMoEQuantConfig to FusedMoEMethodBase subclasses (#22537) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-17 17:43:31 -06:00
bnellnm e9b92dcd89 [Kernels] Overlap shared experts with send/recv (#23273) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-03 12:35:18 -04:00
Xin YangMichael GoinCyrus Leung
 		8fb85b7bb6 Add routed_scaling_factor to MoE grouped topk (#23123) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-08-29 21:36:48 -07:00
Isotr0py 4645024d3a [Quantization] Allow GGUF quantization to skip unquantized layer (#23188) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-08-22 13:04:22 -06:00
bnellnmandMichael Goin 8ad7285ea2 [Kernels] Clean up FusedMoeMethodBase and modular kernel setup. Remove extra arguments from modular kernel methods. (#22035) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-08-15 14:46:00 -04:00
Bowen Wang e9fd658a73 [Feature] Expert Parallelism Load Balancer (EPLB) (#18343) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
 2025-06-26 15:30:21 -07:00
Szymon Ożóg dec66d253b [Kernel] GGUF MMVQ kernel for multiple input vectors (#18754) 
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-06-16 17:33:26 +08:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Isotr0py 1f1b1bc03b [V1][Quantization] Add CUDA graph compatible v1 GGUF support (#18646) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-27 04:40:28 +00:00
Harry Mellor 6223dd8114 Update deprecated type hinting in model_executor/layers (#18056) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 04:17:23 -07:00
Szymon OżógandIsotr0py 1a45a61387 [Kernel] GGUF MoeVec kernel (#16780) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-05-06 23:07:23 -07:00
Harry Mellor 13698db634 Improve configs - ModelConfig (#17130) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-30 10:38:22 +08:00
Lu FangLu FangRoger WangDarkLight1337
 		55dcce91df Upstream Llama4 Support to Main (#16113) 
Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-07 08:06:27 -07:00
LukasBluebaum 90969fb39a [Kernel] Add more dtype support for GGUF dequantization (#15879) 
Signed-off-by: lukas.bluebaum <lukas.bluebaum@aleph-alpha.com>
 2025-04-02 01:58:48 -07:00
Isotr0py 179a619c21 [Bugfix] Fix broken CPU quantization due to triton import (#15038) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-18 08:57:39 -07:00
Szymon Ożóg 55211b01e8 [Bugfix] Fix chunked prefill for GGUF (#14666) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
 2025-03-13 07:19:03 +00:00
Szymon Ożóg e22ee1e7a2 [Kernel] GGUF MoE kernel (#14613) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
 2025-03-12 03:33:27 +00:00
Szymon Ożóg 89cdaa83e7 [Kernel] Add more dtype support for GGUF kernels (#14043) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-03-10 07:30:04 -07:00
Szymon Ożóg 7f0be2aa24 [Model] Deepseek GGUF support (#13167) 2025-02-27 02:08:35 -08:00
Kyle Sayers 12913d17ba [Quant] Add SupportsQuant to phi3 and clip (#13104) 2025-02-15 19:28:33 -08:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Isotr0py b98c62ba49 [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-27 10:43:17 -08:00
Michael Goin 399c798608 Remove ScaledActivation for AWQ (#10057) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-06 14:27:06 +00:00
Isotr0py f19da64871 [Core] Refactor GGUF parameters packing and forwarding (#8859) 2024-10-07 10:01:46 +00:00
Isotr0py fc990f9795 [Bugfix][Kernel] Add IQ1_M quantization implementation to GGUF kernel (#8357) 2024-09-15 16:51:44 -06:00
Isotr0py 7601cb044d [Core] Support tensor parallelism for GGUF quantization (#7520) 2024-08-19 17:30:14 -04:00
Isotr0pyandMichael Goin 360bd67cf0 [Core] Support loading GGUF model (#5191) 
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2024-08-05 17:54:23 -06:00