iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
6,768 Commits 3 Branches 0 Tags
6881107948c00a8564bc2fa85308f6fc2f065d64
Commit Graph
346 Commits
Author SHA1 Message Date
vllmellm d260f799a9 [FEAT] [ROCm] Upgrade AITER Fused MoE kernels. (#18271) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-05-26 23:14:07 -07:00
Isotr0py 1f1b1bc03b [V1][Quantization] Add CUDA graph compatible v1 GGUF support (#18646) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-27 04:40:28 +00:00
Cyrus Leung 503f8487c2 [Misc] Reduce logs on startup (#18649) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-24 23:03:53 -07:00
Wenhua Cheng ec82c3e388 FIX MOE issue in AutoRound format (#18586) 
Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>
 2025-05-23 22:01:40 -07:00
Feng XiaoLongandCrucifixion-Fxl 4fc1bf813a [Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) 
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
 2025-05-23 16:16:26 -07:00
youkaichao 6a7988c55b Refactor pplx init logic to make it modular (prepare for deepep) (#18200) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-05-23 23:43:43 +08:00
Kay Yan 7ab056c273 [Hardware][CPU] Update intel_extension_for_pytorch 2.7.0 and move to requirements/cpu.txt (#18542) 
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
 2025-05-23 04:38:42 -07:00
Michael Goin f4a8a37465 [Minor] Rename quantization nvfp4 to modelopt_fp4 (#18356) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-20 09:08:37 -07:00
Random Fly bca55b556f [Bugfix] fix adding bias twice in ipex GPTQ quantization (#18363) 
Signed-off-by: rand-fly <randfly@outlook.com>
 2025-05-20 00:54:33 -07:00
Wenhua Cheng e2ee1e8e9e [Feature]Add support for models quantized with AutoRound (#17850) 
Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>
 2025-05-19 09:38:53 -07:00
Lain e23564cb70 use ceil_div in cutlass block scaling shape check (#17918) 2025-05-16 03:02:58 -07:00
Jerry Zhang 7974736740 Add support for loading torchao models with AOPerModuleConfig (#17826) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
 2025-05-14 16:24:59 -07:00
bnellnm f9c069c85e Modularize fused experts and integrate PPLX kernels (#15956) 2025-05-14 13:11:54 -07:00
TJianandGregory Shtrasberg 612c2edb4f [FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-14 03:03:11 -07:00
Michael Goin 9a2a6357de [Bugfix] Fix FP8 Marlin MoE and enable for compressed-tensors models (#18026) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-13 19:48:33 -07:00
Pavani Majety 65f0f74b66 [Hardware/NVIDIA/Modelopt] Fix modelopt forward method for v1 torch.compile (#18101) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-05-13 19:33:00 -07:00
vllmellmandtjtanaa 40de1ef455 [FEAT] [ROCm]: Add AITER Block-Scaled GEMM Feature (#14968) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-13 19:08:20 -07:00
Harry Mellor 6223dd8114 Update deprecated type hinting in model_executor/layers (#18056) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-13 04:17:23 -07:00
Michael Goin ea6ae8cb45 [Bugfix] Fix marlin moe fallback logic for llama4 (#18042) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-13 07:53:28 +00:00
Michael Goin 1df491c522 [Bugfix] Fixes for new marlin moe usage (#18017) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-13 03:50:04 +00:00
Michael GoinandDipika 307939f299 Use NVFP4 Marlin for CompressedTensorsW4A16Fp4 (#18000) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika <dipikasikka1@gmail.com>
 2025-05-12 18:07:34 -06:00
Michael Goin f065de4e88 Fix FBGEMM integration (#18002) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-12 23:02:07 +00:00
Dipika Sikka cd3edfc908 [Misc] Add compressed-tensors NVFP4A16 emulation support (#17914) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
 2025-05-11 15:58:38 +08:00
Jinzhen Lin d74e5f37bc [Kernel] fp4 marlin kernel (#17687) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
 2025-05-10 19:58:49 -07:00
Pavani Majety 0c0fdae84f [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) 2025-05-09 16:24:41 -07:00
Michael Goin 22481fbfa3 Update CT WNA16MarlinMoE integration (#16666) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-09 13:19:45 -04:00
Shu Wang 376786fac1 Add cutlass support for blackwell fp8 blockwise gemm (#14383) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
 2025-05-08 15:09:55 -07:00
Michael Goin 4f605a6de5 Fix noisy warning for uncalibrated q_scale/p_scale (#17414) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-08 15:56:59 -04:00
fxmarty-amdandkewang2 bb239a730f [Bugfix] Fix quark fp8 format loading on AMD GPUs (#12612) 
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
 2025-05-08 02:53:53 -07:00
Bowen Bao db593aa67f [Quantization] Quark MXFP4 format loading (#16943) 2025-05-07 15:05:05 -04:00
Szymon OżógandIsotr0py 1a45a61387 [Kernel] GGUF MoeVec kernel (#16780) 
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-05-06 23:07:23 -07:00
Mengqing Cao f9bc5a0693 [Bugfix] Fix triton import with local TritonPlaceholder (#17446) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-05-06 17:53:09 +08:00
Jinzhen Lin 1d0c9d6b2d [Kernel] some optimizations for dense marlin and moe marlin (#16850) 
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
 2025-05-05 09:39:30 -07:00
rasmith e3d0a1d190 [Quantizaton] [AMD] Add support for running DeepSeek int8 w8a8 MoE on ROCm (#17558) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-05-02 21:41:10 -07:00
Eric Hartford 9b103a1d76 fix typo in logging (#17605) 2025-05-02 18:04:40 -07:00
Michael Goin 868c546da4 Support W8A8 INT8 MoE for compressed-tensors (#16745) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-02 10:03:32 -04:00
Michael Goin b4003d11fc Check if bitblas is installed during support check (#17572) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-02 04:32:54 +00:00
Michael Goin 24aebae177 [Bugfix] Disable gptq_bitblas for <SM80 to fix GPTQ on V100/T4 (#17541) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-01 17:59:35 -07:00
NaLan ZeYu 1144a8efe7 [Bugfix] Temporarily disable gptq_bitblas on ROCm (#17411) 
Signed-off-by: Yan Cangang <nalanzeyu@gmail.com>
 2025-04-30 19:51:45 -07:00
Aaron Pham da4e7687b5 [Fix] Support passing args to logger (#17425) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-04-30 08:06:58 -07:00
Harry Mellor 13698db634 Improve configs - ModelConfig (#17130) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-30 10:38:22 +08:00
a2q1p 0ed27ef66c Fix: Spelling of inference (#17387) 2025-04-29 09:23:39 -07:00
Charlie Fu ed2462030f [Bugfix] Fix moe weight losing all extra attrs after process_weights_after_loading. (#16854) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-04-28 21:05:07 +00:00
Harry Mellor c7941cca18 Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:55:31 +00:00
Harry Mellor b6dd32aa07 Make name of compressed-tensors quant method consistent across vLLM (#17255) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-28 16:28:13 +00:00
Charlie Fu 54271bb766 [ROCm][Misc] Follow-ups for Skinny Gemms on ROCm. (#17011) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-04-25 22:05:10 -07:00
rasmith 68af5f6c5c [AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary (#17215) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-04-25 19:55:05 -07:00
rasmithandLuka Govedič a41351f363 [Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-04-25 00:45:02 -07:00
vllmellmandtjtanaa eef364723c [FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-04-25 11:06:50 +08:00
Lei Wangandxinyuxiao 8d32dc603d [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) 
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
 2025-04-22 09:01:36 +01:00