iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
906 Commits
Author SHA1 Message Date
Michael Goin 6e2c176e1f [Bugfix] Restrict Machete to only run on Hopper (#20830) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-12 17:34:40 +00:00
Zhiyu 4afe687a82 Enable ModelOpt Llama4 fp8 checkpoint deployment (#20419) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-07-11 23:07:16 -07:00
Li, Jiang b1235c3e10 [Bugfix] Lazy import fused_experts in BitsAndBytesMoEMethod to avoid break not-cuda-alike devices (#20822) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-11 20:52:05 -07:00
Jee Jee Li 8020e98c9f [Quantization][1/N] MoE support BNB-Inflight Quantization (#20061) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-11 08:01:13 +00:00
Luka Govedičandmgoin 31d5c1797f [Perf][fp8] Use CustomOp abstraction for fp8 quant for better perf (#19830) 
Signed-off-by: Luka Govedic <lgovedic@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-11 04:56:28 +00:00
Wentao Ye e2de455c34 [Feature] Integrate SM100 DeepGEMM support (#20087) 2025-07-10 20:18:05 -07:00
fxmarty-amdandBowen Bao 332d4cb17b [Feature][Quantization] MXFP4 support for MOE models (#17888) 
Signed-off-by: Felix Marty <felmarty@amd.com>
Signed-off-by: Bowen Bao <bowenbao@amd.com>
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Bowen Bao <bowenbao@amd.com>
 2025-07-09 13:19:02 -07:00
Jacob Manning bf03ff3575 [Kernel] Add Conch backend for mixed-precision linear layer (#19818) 
Signed-off-by: Jacob Manning <jmanning+oss@stackav.com>
 2025-07-09 13:17:55 -07:00
Michael Goin 31b96d1c64 Support Llama 4 for cutlass_moe_fp4 (#20453) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-09 15:53:38 -04:00
Ming Yang c438183e99 [Bugfix] Fix topk_ids indices_type for CUTLASS w8a8 FP8 MoE (#20166) 
Signed-off-by: Ming Yang <yming@meta.com>
 2025-07-08 23:10:57 +00:00
Lucia FangandLucia Fang 8aeaa910a2 Fix unknown attribute of topk_indices_dtype in CompressedTensorsW8A8Fp8MoECutlassMethod (#20507) 
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
 2025-07-05 14:03:20 +08:00
Duncan MossandDuncan Moss 3d184b95b8 [feat]: CUTLASS block scaled group gemm for SM100 (#19757) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Co-authored-by: Duncan Moss <dmoss@nvidia.com>
 2025-07-04 12:58:04 -06:00
Michael Goin 0e3fe896e2 Support Llama 4 for fused_marlin_moe (#20457) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-04 07:55:10 +00:00
Jee Jee Li 1caca5a589 [Misc] Add SPDX-FileCopyrightText (#20428) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-04 07:40:42 +00:00
bnellnm 78fe77534b [Kernel] Enable fp8 support for pplx and BatchedTritonExperts. (#18864) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-07-03 14:55:40 -07:00
Jee Jee Li 1819fbda63 [Quantization] Bump to use latest bitsandbytes (#20424) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-03 21:58:46 +08:00
Li, Jiang 0ec3779df7 [Bugfix][CI/CD][CPU] Fix CPU CI tests (#20383) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-02 20:11:36 -07:00
bnellnm 2e25bb12a8 [Bugfix] Fix import of CutlassExpertsFp8 in compressed_tensors_moe.py (#20381) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-07-03 02:07:43 +00:00
bnellnmandElizaWszola c1909e7e8c [Kernels] MoE refactor (#19636) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2025-07-02 06:08:27 -07:00
czhu-cohere 3abfe22154 Enable group size 64 for Machete (#20290) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-07-01 18:05:44 -07:00
TJian 02cabff207 [V1] [ROCm] Enable EP with AITER Fused MoE (#20270) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-07-01 16:48:30 +00:00
Kyle Sayers 9025a9a705 [Quant] [Bugfix] Fix quantization config matching with hf_to_vllm_mapper (#20046) 2025-07-01 19:20:34 +09:00
czhu-cohere 9909726d2a Enable ZP Support for Machete (#20268) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-07-01 07:12:20 +00:00
Alex KoganandMichael Goin 27949354fa [Feature] A calibration-free RTN-based quantization for accurate and accelerated INT4/INT8 inference (#18768) 
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-07-01 05:44:38 +00:00
li haoyangandHaoyang Li 1c50e100a9 [Bugfix] fix quark ptpc (#20251) 
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: Haoyang Li <307790822@qq.com>
 2025-06-30 22:24:50 +09:00
Dipika Sikka 6f2f53a82d [Quantization] Add compressed-tensors NVFP4 MoE Support (#19990) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
 2025-06-29 22:05:40 +00:00
Wentao Ye 4d36693687 [Refactor] Create a function util and cache the results for has_deepgemm, has_deepep, has_pplx (#20187) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-28 22:06:38 +00:00
Bowen Wang e9fd658a73 [Feature] Expert Parallelism Load Balancer (EPLB) (#18343) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
 2025-06-26 15:30:21 -07:00
Li, Jiang 0567c8249f [CPU] Fix torch version in x86 CPU backend (#19258) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-06-26 03:34:47 -07:00
Dipika Sikka 02c97d9a92 [Quantization] Add compressed-tensors emulations support for NVFP4 (#19879) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
 2025-06-25 14:28:19 -04:00
Wentao Ye 879f69bed3 [Refactor] Remove duplicate ceil_div (#20023) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-25 05:19:09 +00:00
Wentao Ye a6c4b87fbc Revert "[Feature] Integrate new deepgemm (#19820)" (#20049) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 19:45:22 -07:00
Boyuan Feng c01d1c5aba use .dev for version comparison with pytorch nightly release (#20031) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-06-24 21:52:16 +00:00
Wentao Ye c6e3bba8e6 [Feature] Integrate new deepgemm (#19820) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 12:51:56 -07:00
Jun-Howie dd2ccf8dde Feat Dynamic Quantization for MoE Layers in GPTQ Marlin Backend (#19395) 2025-06-24 07:23:28 +09:00
Dipika Sikka 6bc7b57315 [Quantization] Remove FP4 emulation; Fall-back to marlin for device < 100 (#19563) 2025-06-16 17:33:51 -04:00
Szymon Ożóg dec66d253b [Kernel] GGUF MMVQ kernel for multiple input vectors (#18754) 
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-06-16 17:33:26 +08:00
Boyuan Feng ce688ad46e use base version for version comparison (#19587) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-06-13 15:09:34 +08:00
mobicham 96846bb360 Fix TorchAOConfig skip layers (#19265) 
Signed-off-by: mobicham <hicham@mobiuslabs.com>
 2025-06-12 22:22:53 +08:00
Jee Jee Li 73e2e0118f [Quantization] Improve AWQ logic (#19431) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-12 11:02:11 +00:00
rasmith 2e090bd5df [AMD][Kernel][BugFix] fix test_rocm_compressed_tensors_w8a8 for rocm (#19509) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-06-12 07:14:24 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath e5d35d62f5 [BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-12 04:28:12 +00:00
Ning Xie 2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
bnellnm 29fa5cac1c [Kernels] Add activation chunking logic to FusedMoEModularKernel (#19168) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-11 12:53:10 -04:00
artetaout b8e809a057 [Kernel] Support deep_gemm for linear methods (#19085) 
Signed-off-by: artetaout <lulala341@gmail.com>
 2025-06-11 15:14:45 +08:00
Jee Jee LiIsotr0pygemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		b6553be1bc [Misc] Slight improvement of the BNB (#19418) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-06-10 13:51:49 +00:00
Varun Sundar RabindranathandVarun 5cf2daea9a [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-09 10:50:39 -04:00
Dipika Sikka c123bc33f9 [Quantization] Add compressed-tensors NVFP4 support (#18312) 2025-06-08 09:05:55 -04:00
ElizaWszolaandTyler Michael Smith 84166fee97 [Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-06-06 18:26:11 -07:00
Dipika Sikka 94870359cd [Quantization] Bump compressed-tensors version; update NVFP4A16 test model (#19224) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
 2025-06-06 01:21:54 -07:00