iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
7,169 Commits 3 Branches 0 Tags
93aee29fdb81fa51d5d44f34ea84df80d0764235
Commit Graph
814 Commits
Author SHA1 Message Date
Di Liu 5b3ad5ecf2 [DOC] fix doc typos (#19600) 
Signed-off-by: Di Liu <liu-di@sjtu.edu.cn>
 2025-06-17 11:34:53 +08:00
Dipika Sikka 6bc7b57315 [Quantization] Remove FP4 emulation; Fall-back to marlin for device < 100 (#19563) 2025-06-16 17:33:51 -04:00
bnellnm 5e5baa91aa [Kernels] Use empty for modular MoE workspaces (#19667) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-16 14:58:01 +00:00
Szymon Ożóg dec66d253b [Kernel] GGUF MMVQ kernel for multiple input vectors (#18754) 
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
 2025-06-16 17:33:26 +08:00
wang.yuqi f40f763f12 [CI] Add mteb testing for rerank models (#19344) 2025-06-16 01:36:43 -07:00
Boyuan Feng ce688ad46e use base version for version comparison (#19587) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-06-13 15:09:34 +08:00
Varun Sundar RabindranathandVarun Sundar Rabindranath e3b12667d4 [BugFix] : Fix Batched DeepGemm Experts (#19515) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-12 20:43:02 -06:00
Varun Sundar Rabindranath 9d880f594d [Misc] Turn MOE_DP_CHUNK_SIZE into an env var (#19506) 2025-06-12 18:01:16 +00:00
mobicham 96846bb360 Fix TorchAOConfig skip layers (#19265) 
Signed-off-by: mobicham <hicham@mobiuslabs.com>
 2025-06-12 22:22:53 +08:00
Jee Jee Li 73e2e0118f [Quantization] Improve AWQ logic (#19431) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-12 11:02:11 +00:00
rasmith 2e090bd5df [AMD][Kernel][BugFix] fix test_rocm_compressed_tensors_w8a8 for rocm (#19509) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-06-12 07:14:24 +00:00
Brayden Zhong 3f6341bf7f Add Triton Fused MoE kernel config for E=16 on B200 (#19518) 
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2025-06-12 04:31:51 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath e5d35d62f5 [BugFix] Force registration of w8a8_block_fp8_matmul_deepgemm via lazy import (#19514) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-12 04:28:12 +00:00
Ning Xie 2f1c19b245 [CI] change spell checker from codespell to typos (#18711) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-06-11 19:57:10 -07:00
bnellnm 29fa5cac1c [Kernels] Add activation chunking logic to FusedMoEModularKernel (#19168) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-11 12:53:10 -04:00
Ximingwang-09andximing.wxm 3c8694eabe Fix some typo (#19475) 
Signed-off-by: ximing.wxm <ximing.wxm@antgroup.com>
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
 2025-06-11 10:36:04 +00:00
artetaout b8e809a057 [Kernel] Support deep_gemm for linear methods (#19085) 
Signed-off-by: artetaout <lulala341@gmail.com>
 2025-06-11 15:14:45 +08:00
Junhao Li 2d40665fe8 Add fused MOE config for Qwen3 30B A3B on B200 (#19455) 
Signed-off-by: Junhao Li <junhao@ubicloud.com>
 2025-06-11 13:43:46 +08:00
Xu Wenqing 22c3c0aa4a Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B-FP8 (#19401) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-06-11 07:23:57 +08:00
Jee Jee LiIsotr0pygemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		b6553be1bc [Misc] Slight improvement of the BNB (#19418) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-06-10 13:51:49 +00:00
Varun Sundar RabindranathandVarun 5cf2daea9a [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-09 10:50:39 -04:00
Dipika Sikka c123bc33f9 [Quantization] Add compressed-tensors NVFP4 support (#18312) 2025-06-08 09:05:55 -04:00
Xu Wenqing 989dcee981 Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B (#19315) 
Signed-off-by: Xu Wenqing <xuwq1993@qq.com>
 2025-06-08 16:07:02 +08:00
ElizaWszolaandTyler Michael Smith 84166fee97 [Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-06-06 18:26:11 -07:00
Dipika Sikka 94870359cd [Quantization] Bump compressed-tensors version; update NVFP4A16 test model (#19224) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
 2025-06-06 01:21:54 -07:00
Jerry Zhang c8134bea15 Fix AOPerModuleConfig name changes (#18869) 
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
 2025-06-05 18:51:32 -07:00
Chiyue WeiandChiyue Wei 61059bee40 [Hardware][NVIDIA] FP4 MoE kernel optimization (#19110) 
Signed-off-by: Chiyue Wei <chiyuew@nvidia.com>
Co-authored-by: Chiyue Wei <chiyuew@nvidia.com>
 2025-06-05 09:48:26 -07:00
Xu Wenqing ec89524f50 Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 (#19205) 2025-06-05 16:38:54 +00:00
Varun Sundar RabindranathandVarun c3fd4d669a [Kernel] Integrate batched/masked deepgemm kernel (#19111) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-04 21:59:18 +00:00
Lain 5f2cd251d2 Sm100 blockwise fp8 swap ab (#18564) 2025-06-04 07:48:45 -07:00
Vadim Gimpelson 5d6d1adf15 [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437) 2025-06-03 21:13:01 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath fa98d77773 [Kernel] DeepEP dispatch-combine kernel integration (#18434) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-03 12:30:02 -07:00
Simon Mo 02f0c7b220 [Misc] Add SPDX-FileCopyrightText (#19100) 
Signed-off-by: simon-mo <simon.mo@hey.com>
 2025-06-03 11:20:17 -07:00
Tyler Michael Smith 8a57872b2a [Bugfix][EP+DP] Use pplx-kernel internode instead of intranode (#19034) 
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-06-03 11:36:51 +08:00
Charlie Fu 306d60401d [ROCm][Kernel] Add gfx950 support for skinny gemms (#18010) 
Signed-off-by: charlifu <charlifu@amd.com>
 2025-05-31 07:40:05 -07:00
vllmellm 0f5e0d567e [FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 (#18825) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-05-31 03:39:31 -07:00
rongfu.leng 7f21e8052b [Misc] add group_size is -1 in awq quantization (#18910) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-05-30 17:34:22 +00:00
Michael Goin 4d0a1541be [Bugfix] Remove NVFP4 scales assertions to fix load_format=dummy (#18861) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-30 13:37:36 +08:00
Wenhua Cheng 3de3eadf5b improve the robustness of parsing vlms config in AutoRound (#18894) 
Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>
 2025-05-29 19:24:47 -07:00
Cyrus Leung 1aa2f81b43 [Misc] Update type annotation for rotary embedding base (#18914) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-30 10:17:01 +08:00
Chengji Yao a1cc9f33a3 [TPU] remove transpose ops in moe kernel (#18923) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-05-29 23:00:11 +00:00
Chengji Yao 8e882ffdc0 [Bugfix][TPU] fix moe custom kernel import (#18853) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-05-28 19:34:19 -07:00
Maximilien de BayserandCyrus Leung 515b413ebf Prevent the cross-encoder logic from being applied to classification tasks (#18838) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-05-28 19:16:17 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 7951d78738 [Core] Enable CUDA graphs for DP + All2All kernels (#18724) 
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
 2025-05-28 22:55:30 +00:00
Satyajith Chilappagari e0cbad4e30 [Neuron] Support quantization on neuron (#18283) 
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
 2025-05-27 22:10:33 +00:00
Hyogeun Oh (오효근) a68e293cb9 [Doc] Convert Sphinx directives ( {class}, {meth}, {attr}, ...) to MkDocs format for better documentation linking (#18663) 
Signed-off-by: Zerohertz <ohg3417@gmail.com>
 2025-05-27 01:44:20 -07:00
almersawiandIslam Almersawi a547aeb828 feat(rocm-support): support mamba2 on rocm (#18565) 
Signed-off-by: Islam Almersawi <islam.almersawi@openinnovation.ai>
Co-authored-by: Islam Almersawi <islam.almersawi@openinnovation.ai>
 2025-05-27 00:07:53 -07:00
vllmellm d260f799a9 [FEAT] [ROCm] Upgrade AITER Fused MoE kernels. (#18271) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-05-26 23:14:07 -07:00
Isotr0py 1f1b1bc03b [V1][Quantization] Add CUDA graph compatible v1 GGUF support (#18646) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-27 04:40:28 +00:00
Cyrus Leung 503f8487c2 [Misc] Reduce logs on startup (#18649) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-05-24 23:03:53 -07:00