iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
6,768 Commits 3 Branches 0 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
768 Commits
Author SHA1 Message Date
Gregory Shtrasbergandcharlifu f533b5837f [ROCm][Kernel] MoE weights padding (#14454) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
 2025-03-24 23:45:30 +00:00
Gregory Shtrasberg 8279201ce6 [Build] Cython compilation support fix (#14296) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-03-24 23:37:54 +00:00
liuzhenwei 5eeadc2642 [Hardware][Gaudi][Feature] Enable Dynamic MoE for Mixtral (#12303) 
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
 2025-03-24 09:48:40 -07:00
Jee Jee Li 3892e58ad7 [Misc] Upgrade BNB version (#15183) 2025-03-24 05:51:42 +00:00
Qubitium-ModelCloud d20e261199 Fix non-contiguous input passed to Marlin kernel (#15319) 2025-03-24 03:09:44 +00:00
DefTruth f90d34b498 [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322) 
Signed-off-by: DefTruth <qiustudent_r@163.com>
 2025-03-23 01:10:10 -07:00
TJian ec870fba9a [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-03-21 22:36:14 -07:00
Chih-Chieh Yang 296f927f24 [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-03-20 19:21:08 -07:00
Yu Chin Fabian Lim 06dd08256f Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617) 
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
 2025-03-21 00:44:37 +00:00
Mickaël Seznec a597a57595 [Attention] Flash Attention 3 - fp8 (#14570) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
 2025-03-20 01:14:20 -04:00
Isotr0py 179a619c21 [Bugfix] Fix broken CPU quantization due to triton import (#15038) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-18 08:57:39 -07:00
yury-tokpanovQuentin AnthonyTyler Michael SmithCyrus Leung
 		452e8fd968 [MODEL] Add support for Zamba2 models (#13185) 
Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-18 08:56:21 -07:00
ekuznetsov139 8b793f7ec6 MI325 configs, fused_moe_kernel bugfix (#14987) 
Signed-off-by: Eugene Kuznetsov <eugene.kuznetsov@amd.com>
 2025-03-18 08:05:18 -07:00
Simon Mo 71c1e07107 [Kernel] Add more tuned configs (#14877) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-03-15 20:25:03 -07:00
Tyler Michael Smith ccf02fcbae Revert "[Model] Mamba2 Prefill Performance Tweaks: Fixing Flurry of U… (#14848) 2025-03-14 20:45:42 -07:00
Chih-Chieh Yang fe66b34728 [Model] Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14778) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-03-14 16:36:18 -04:00
Lucas Wilkinson 9532c49836 [Attention] MLA get rid of materialization (#14770) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-13 23:39:02 -07:00
Thien Tran 95d680b862 [Bugfix][IPEX] Add VLLM_CPU_MOE_PREPACK to allow disabling MoE prepack when CPU does not support it (#14681) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2025-03-13 20:43:18 -07:00
Lucas Wilkinson d47807ba08 [Attention] Remove slow setattr in MLA (#14769) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-13 21:31:14 +00:00
Szymon Ożóg 55211b01e8 [Bugfix] Fix chunked prefill for GGUF (#14666) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
 2025-03-13 07:19:03 +00:00
Gregory Shtrasberg a94a699c3f [ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-03-12 20:14:04 -07:00
Li, JiangandIsotr0py ff47aab056 [CPU] Upgrade CPU backend to torch-2.6 (#13381) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-03-12 10:41:13 +00:00
Pavani Majety debd6bbf09 [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-03-12 05:13:11 +00:00
Szymon Ożóg e22ee1e7a2 [Kernel] GGUF MoE kernel (#14613) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
 2025-03-12 03:33:27 +00:00
Isotr0py e392d85831 [Core] Refactor QKVCrossParallelLinear implementation to support BNB 4-bit quantization (#14545) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-11 20:12:52 -07:00
Jeff Daily a1c8f3796c dynamic distpatch of fp8 kernels (#14245) 
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
 2025-03-11 10:54:56 -04:00
Jinzhen Linandmgoin 90e88ab756 [Kernel] moe wna16 cuda kernel (#13321) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-03-10 20:12:40 -04:00
Roger Wang dea985aef0 [V1][Bugfix] Fix handing of second_per_grid_ts for Qwen2-VL & Qwen2.5-VL (#14548) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-10 16:03:11 +00:00
Szymon Ożóg 89cdaa83e7 [Kernel] Add more dtype support for GGUF kernels (#14043) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-03-10 07:30:04 -07:00
Tyler Michael Smith ca7a2d5f28 Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) 2025-03-07 22:18:53 -08:00
Jinzhen Lin d0feea31c7 [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
 2025-03-07 11:53:38 -05:00
Luka Govedič e1744502c2 [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-03-07 05:20:16 +00:00
Lucas Wilkinson dae6896977 [Perf] Reduce MLA CPU overheads in V1 (#14384) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-03-06 19:59:14 -08:00
Tyler Michael Smith 958adce478 [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-06 14:17:21 -08:00
Tyler Michael Smith cc2f9b32c8 [Distributed] Add enable_expert_parallel arg (#14305) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-06 18:54:45 +00:00
Nicolò LucchesiandNickLucche 69ff99fdcd [Core] Optimizing cross-attention QKVParallelLinear computation (#12325) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
 2025-03-06 09:37:26 +00:00
Isotr0py e17e4488bd [LoRA] Remove linear hack outside transformers backend (#14177) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-05 15:06:28 +00:00
Tyler Michael Smith 72c62eae5f [V1] EP/TP MoE + DP Attention (#13931) 2025-03-04 21:27:26 -08:00
rainkertanddangshunya 0df25101d6 [Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) 
Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>
 2025-03-05 12:25:53 +08:00
Divakar Verma bb5b640359 [core] moe fp8 block quant tuning support (#14068) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2025-03-04 01:30:23 +00:00
Michael Goin 19d98e0c7d [Kernel] Optimize moe intermediate_cache usage (#13625) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-03-03 16:29:53 -05:00
Mengqing Cao b87c21fc89 [Misc][Platform] Move use allgather to platform (#14010) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-03-03 15:40:04 +08:00
YajieWang 6a92ff93e1 [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) 2025-02-28 22:30:59 -08:00
Luka Govedič bd56c983d6 [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-02-28 16:20:11 -07:00
Jee Jee Li e0734387fb [Bugfix] Fix MoeWNA16Method activation (#14024) 2025-02-28 15:22:42 +00:00
Jee Jee Li 67fc426845 [Misc] Print FusedMoE detail info (#13974) 2025-02-27 18:53:13 -05:00
qli88 8294773e48 [core] Perf improvement for DSv3 on AMD GPUs (#13718) 
Signed-off-by: qli88 <qiang.li2@amd.com>
 2025-02-27 22:14:30 +00:00
Szymon Ożóg 7f0be2aa24 [Model] Deepseek GGUF support (#13167) 2025-02-27 02:08:35 -08:00
Woosuk Kwon b382a7f28f [BugFix] Make FP8 Linear compatible with torch.compile (#13918) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-02-26 13:48:55 -08:00
Michael Goin 07c4353057 [Model] Support Grok1 (#13795) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-02-26 01:07:12 +00:00