iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
8,184 Commits 3 Branches 0 Tags
0f46a780d4f53b8564a37370f9f068cdf4e69604
Commit Graph
960 Commits
Author SHA1 Message Date
Wentao Ye c3e0e9337e [Feature] Add Flashinfer MoE Support for Compressed Tensor NVFP4 (#21639) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-31 15:26:11 -07:00
amirkl94andmgoin 207b750e19 [NVIDIA] Add SM100 Flashinfer MoE per tensor scale fp8 backend (#21458) 
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-31 06:00:01 -07:00
Jee Jee Li 0f7919fca0 [Misc] Expand SUPPORTED_HIDDEN_SIZES for DeepEP low-latency kernels (#21818) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-30 20:41:12 -07:00
Po-Han Huang (NVIDIA) ff08e51940 [NVIDIA] Fix Llama4 Scout FP4 functionality issues (#21499) 
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
 2025-07-30 07:33:40 -07:00
Wenhua Cheng ad341c5194 [Bugfix]fix mixed bits and visual language model quantization in AutoRound (#21802) 
Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>
 2025-07-29 07:26:31 -07:00
Isotr0py a4528f0cac [Model]: Fused MoE for nomic-embed-text-v2-moe (#18321) 
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-29 03:13:27 -07:00
Wentao Ye 48b763d6b5 [Refactor] Merge Compressed Tensor FP8 CompressedTensorsW8A8Fp8MoEMethod and CompressedTensorsW8A8Fp8MoECutlassMethod (#21775) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-28 19:47:21 -06:00
Nikhil Guptaandmgoin 89ac266b26 [Feat]: Add support for Dynamic Quant 4 bit CPU kleidiai kernels (#17112) 
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-28 20:55:15 +00:00
rasmith b361f14e39 [AMD][BugFix] Fix omission of wvSplitK kernel for small batch sizes (1-4) due to torch.compile (#21350) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-07-28 15:38:20 -04:00
Asaf Joseph Gardinandasafg a6c050286a [v1][mamba] Added mamba_type into MambaSpec (#21715) 
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-07-28 08:15:55 +00:00
TJian e626d286f5 [FEAT] [ROCm] [AITER]: Add AITER HIP block quant kernel (#21242) 2025-07-28 05:07:06 +00:00
Jee Jee Li 04ff4be310 [Misc] Add fused_moe configs for Qwen3-Coder-480B-A35B-Instruct-FP8 (#21700) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-27 20:12:18 -07:00
Caleb_Du 57c22e57f9 Fix CUDA permute/unpermute for use with DeepGemm Moe (#17934) 
Signed-off-by: Caleb_Du <Caleb_Du@zju.edu.cn>
 2025-07-27 07:08:00 -07:00
Wentao Ye bda9d0535f [Refactor] Refactor MOE NVFP4 Code Base: ModelOpt + Compressed Tensor (#21631) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-27 05:25:21 -07:00
Kaixi Hou de509ae8eb [NVIDIA] Explicitly disable shuffled weights for flashinfer blockscale moe fp8 kernels (#21411) 
Signed-off-by: kaixih <kaixih@nvidia.com>
 2025-07-26 07:10:36 -07:00
Wentao Ye 56e544f24b [Refactor] Remove moe_align_block_size_triton (#21335) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-26 07:08:29 -07:00
Alex Kogan 7ae75fa6d0 [Feature] Add support for MoE models in the calibration-free RTN-based quantization (#20766) 
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
 2025-07-25 18:09:34 -07:00
Wentao Ye 75d29cf4e1 [Perf] Cuda Kernel for Int8 Per Token Group Quant (#21476) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-25 17:07:07 -07:00
Chih-Chieh YangandYu Chin Fabian Lim eab2f3980c [Model] Replace Mamba2 RMSNorm Gated with Fused Triton Kernel (#20839) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Signed-off-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
Signed-off-by: Chih-Chieh Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
 2025-07-25 06:49:36 -07:00
Cyrus Leung 46d81d6951 [V1] Get supported tasks from model runner instead of model config (#21585) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-25 05:36:45 -07:00
Xu Wenqing 8ed01e32f7 Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#21598) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-07-25 02:36:55 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 2212cd6cfb [Bugfix] DeepGemm utils : Fix hardcoded type-cast (#21517) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-24 20:17:29 -07:00
Burkhard RingleinandThomas Parnell ce3a9b1378 [Kernel] adding fused_moe configs for upcoming granite4 (#21332) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-24 20:16:59 -07:00
Wentao Ye 633f6e804b [Bug] Fix DeepGemm Init Error (#21554) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-24 20:07:22 -07:00
Woosuk Kwon fe56180c7f [MoE] More balanced expert sharding (#21497) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-07-24 15:56:08 -07:00
Shu Wang 1b25f1fe75 Update flashinfer CUTLASS MoE Kernel (#21408) 
Signed-off-by: Shu Wang. <shuw@nvidia.com>
 2025-07-24 08:13:31 -07:00
Nick Hill f0f4de8f26 [Misc] Fix duplicate FusedMoEConfig debug messages (#21455) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-24 01:27:30 -07:00
Chengji YaoandSimon Mo e74bfc70e4 [TPU][Bugfix] fix moe layer (#21340) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-07-24 00:38:39 -07:00
Michael Goin f002e9a870 [Cleanup] Only log MoE DP setup warning if DP is enabled (#21315) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-23 00:02:48 -07:00
Wentao Ye 774d0c014b [Perf] Cuda Kernel for Per Token Group Quant (#21083) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-22 07:27:15 -07:00
Duncan Mossjiahancmgoin
 		2c8db17cfd [feat]: add SM100 support for cutlass FP8 groupGEMM (#20447) 
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-22 07:27:12 -07:00
Mickaël Seznecandmgoin 4fb56914c5 [perf] Add fused MLA QKV + strided layernorm (#21116) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-22 07:07:44 -07:00
Shu Wang 9e23ad9655 Update fp4 quantize API (#21327) 
Signed-off-by: Shu Wang <shuw@nvidia.com>
 2025-07-21 23:40:21 -07:00
Ming Yang e7b2042681 Revert "[Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) (#21334) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-07-21 21:49:01 -07:00
Himanshu Jaju 0ec82edda5 [perf] Speed up align sum kernels (#21079) 
Signed-off-by: Himanshu Jaju <hj@mistral.ai>
 2025-07-21 11:19:23 -07:00
Zhiyu 6b46c4b653 Add Nvidia ModelOpt config adaptation (#19815) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
 2025-07-21 10:02:58 -04:00
Cyrus Leung 042af0c8d3 [Model][1/N] Support multiple poolers at model level (#21227) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-21 02:22:21 -07:00
Thomas Parnell 881e3cbe3b [V1] [Hybrid] Enable piecewise CUDA Graph for mamba layers (#21194) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-19 19:27:21 +00:00
Kaixi Houandmgoin 6d0734c562 [NVIDIA] Add SM100 Flashinfer MoE blockscale fp8 backend for low latency (#20645) 
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-19 02:33:01 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath dcc6cfb991 [Kernel][Performance] Tweak MoE Batched silu_mul_fp8_quant_deep_gemm kernel (#21193) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-18 23:09:51 -07:00
Woosuk Kwon dd572c0ab3 [V0 Deprecation] Remove V0 Spec Decode workers (#21152) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-18 21:47:50 -07:00
Rui Qiao 217937221b Elastic Expert Parallel Initial Support (#20775) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-18 17:46:09 -07:00
Richard Zou b2eb2b5ad7 [Kernel] Apply torch.Tag.needs_fixed_stride_order only for torch==2.6.0 (#19346) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-07-18 14:10:21 -04:00
Cyrus Leung 45badd05d0 [Core] Set pooling params based on task and model (#21128) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-18 05:41:17 -07:00
ElizaWszola 4adc66f64d [Bugfix] Allocate less memory in non-batched CUTLASS MoE (#21121) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-18 18:55:52 +08:00
Shu Wangandmgoin c7d8724e78 [Core] FlashInfer CUTLASS fused MoE backend (NVFP4) (#20037) 
Signed-off-by: shuw <shuw@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-17 21:32:45 -07:00
Wentao Ye 8a8fc94639 [Log] Debugging Log with more Information (#20770) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-18 00:19:46 +00:00
Woosuk Kwon 4de7146351 [V0 deprecation] Remove V0 HPU backend (#21131) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-17 16:37:36 -07:00
Cyrus Leung 90bd2ab6e3 [Model] Update pooling model interface (#21058) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-17 16:05:40 +00:00
ElizaWszola 9fb2d22032 [Performance] Performance improvements in non-blockwise fp8 CUTLASS MoE (#20762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
 2025-07-17 09:56:44 -04:00