iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,845 Commits 3 Branches 0 Tags
78300cdabffbaaffe7625ed34412ec2aec75535c
Commit Graph
44 Commits
Author SHA1 Message Date
fxmarty-amdandAndreas Karatzas 63988f3c2d [Quantization][Refactor][1/N] Adopt QuantKey in QuarkConfig and methods, relying on weight_quant_key, act_quant_key for quant method dispatch (#52958) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-01 02:55:54 -05:00
ErenAta16 fbb17e780b [Bugfix] Fix six quantization exception messages split across positional args (#50479) 
Signed-off-by: ErenAta16 <erena6466@gmail.com>
 2026-08-21 09:06:56 -07:00
fxmarty-amd 88b2bff2c6 [MOE] Standardize and abstract fused shared expert optimization selection (#51695) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-18 12:56:56 -04:00
fxmarty-amd 48825acc23 [Quantization] Remove dead QuantizationConfig.is_mxfp4_quant (#51793) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-08-13 13:58:31 -04:00
jimmy-adamsAndreas KaratzashujiCursor
 		da788334bc Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel (#47972) 
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: huji <huidong.ji@amd.com>
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: huji <huidong.ji@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-07 01:19:31 -07:00
xiao-llmandAndreas Karatzas 02bf9c7907 Fix Quark mxfp4 quantized model loading issue under mtp (#46757) 
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-16 14:58:47 -04:00
Harry Mellor 5051698e41 Remove unnecessary load_weights methods (#44589) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-29 01:52:23 -07:00
Shantipriya Parida a1ec011a83 [Bugfix] Add deepseek_v32 to Quark dynamic MXFP4 model type check (#39498) 
Signed-off-by: Shantipriya Parida <shantipriya.parida@amd.com>
 2026-06-10 02:52:33 -07:00
Harry Mellor a80af24356 Speed up docs build (#44635) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-05 14:51:44 +00:00
Harry Mellorandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 62215e72c6 Remove KV cache scale boilerplate from model weight loading methods (#43167) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-05 05:19:04 -07:00
fxmarty-amdandKyle Sayers 40330967ab [Quark] Support loading Quark NVFP4 checkpoints in vLLM (#35859) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-05-13 11:17:36 -07:00
bnellnmRobert ShawRobert Shaw
 		4d591db470 [MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK (#40735) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-05-12 13:37:44 -04:00
Hemanth Acharya 6841f5dc77 [ROCm] Add env flags to disable dynamic MXFP4 quant and enable AITER tuned GEMMs for Attention Projection Layers (#39987) 
Signed-off-by: Hemanth Acharya <heachary@amd.com>
 2026-04-29 16:07:46 -07:00
Martin Hickey 3951d3eacd [MyPy] Enable mypy for vllm/model_executor/layers/ (#40159) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-04-21 20:15:02 -07:00
PikaPikachu 827268e98d [Quantization] Support Quark W8A8 INT8 MoE inference (#36320) 
Signed-off-by: kangletian <Letian.Kang@amd.com>
 2026-04-09 17:24:43 +00:00
Hongxia YangandHongxia Yang dbdd9ae067 [ROCm][Bugfix] fix exception related to trust_remote_code for MiniMax-M2.1-MXFP4 (#37698) 
Signed-off-by: Hongxia Yang <hongxiay.yang@amd.com>
Co-authored-by: Hongxia Yang <hongxiay.yang@amd.com>
 2026-03-30 15:49:23 +00:00
xiao-llmYanwen LinKim Hee Su
 		24b4272a8c Fix infinite recursive search issue in quark.py (#32779) 
Signed-off-by: Yanwen Lin <lyw1124278064@gmail.com>
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Signed-off-by: kimheesu <wlskaka4@gmail.com>
Co-authored-by: Yanwen Lin <lyw1124278064@gmail.com>
Co-authored-by: Kim Hee Su <wlskaka4@gmail.com>
 2026-03-17 07:19:15 +00:00
Divakar Verma 6341d43043 [ROCm][Quantization] add quark w4a8 mxfp4_fp8 for LinearLayer (#35316) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-03-13 19:44:24 +00:00
Douglas LehrDoug LehrRohan PotdarGregory Shtrasberg
 		ec8ab9d254 [ROCm] Add dynamic mxfp4 quantization for DeepSeek V2 projection layers (#34157) 
Signed-off-by: Doug Lehr <douglehr@amd.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
 2026-02-26 10:00:49 -06:00
xuebwang-amdandTJian 766e167821 [ROCm][quantization] improve OCP weight quant parser robust (#34431) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-02-12 09:40:19 -08:00
xuebwang-amdRobert ShawRobert Shaw
 		b129136c7a [ROCm][Quantization] GPT_OSS in amd-quark format model loading and emulations (#29008) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-10 10:08:05 -05:00
杨朱 · KikiandClaude Opus 4.5 1a7894dbdf [Misc] Replace Optional[X] with X | None syntax (#33332) 
Signed-off-by: carlory <baofa.fan@daocloud.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-30 01:56:59 -08:00
Matthew Bonanni a608b4c6c2 [5/N][Attention] Finish eliminating vllm/attention folder (#32064) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-27 10:02:51 -05:00
Bowen Bao 0c738b58bc [Quantization] Support Quark int4-fp8 w4a8 for MoE (#30071) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2025-12-18 04:20:42 +00:00
Matthew Bonanni fc1d8be3dc [Attention] Update attention imports (#29540) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-27 11:19:09 -05:00
xuebwang-amdandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d0a73620cc [ROCm][Quantization] add apply_vllm_mapper in quark config for models like gpt-oss (#28638) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-18 11:16:45 +08:00
haoyangli-amd 0b25498990 [Misc] add ignore mapper for quark quantization (#28275) 
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
 2025-11-14 05:56:35 +00:00
xuebwang-amdfxmarty-amdCyrus Leung
 		05576df85c [ROCm][Quantization] extend AMD Quark to support mixed-precision quantized model (#24239) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Co-authored-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-11-11 12:05:22 -05:00
Harry Mellor 8fcaaf6a16 Update Optional[x] -> x | None and Union[x, y] to x | y (#26633) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-12 09:51:31 -07:00
fxmarty-amd 41f1cf38f2 [Feature][OCP MX] Support mxfp6 and mixed mxfp6-mxfp4 (#21166) 2025-10-07 09:35:26 -04:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Aleksandr MalyshevAleksandr MalyshevDoug Lehr
 		53a30845be Llamas 3.1 405B fp4 changes upstreaming from 355_wip (#25135) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
 2025-09-25 19:16:53 -06:00
fxmarty-amdandBowen Bao 332d4cb17b [Feature][Quantization] MXFP4 support for MOE models (#17888) 
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>
 2025-07-09 13:19:02 -07:00
li haoyangandHaoyang Li 1c50e100a9 [Bugfix] fix quark ptpc (#20251) 
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: Haoyang Li <307790822@qq.com>
 2025-06-30 22:24:50 +09:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Harry Mellor 6223dd8114 Update deprecated type hinting in model_executor/layers (#18056) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 04:17:23 -07:00
Bowen Bao db593aa67f [Quantization] Quark MXFP4 format loading (#16943) 2025-05-07 15:05:05 -04:00
Harry Mellor 13698db634 Improve configs - ModelConfig (#17130) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-30 10:38:22 +08:00
rasmith 68af5f6c5c [AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary (#17215) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-04-25 19:55:05 -07:00
rasmithandLuka Govedič a41351f363 [Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-04-25 00:45:02 -07:00
Kyle Sayers 12913d17ba [Quant] Add SupportsQuant to phi3 and clip (#13104) 2025-02-15 19:28:33 -08:00
Kyle Sayersandmgoin 7ff7a638b6 [Model][Quant] Fix GLM, Fix fused module mappings for quantization (#12634) 
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
 2025-02-05 05:32:06 +00:00
Russell Bryant e489ad7a21 [Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
kewang-xlnxkewang2Michael Goin
 		de0526f668 [Misc][Quark] Upstream Quark format to VLLM (#10765) 
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
 2025-01-15 11:05:15 -05:00