iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
1981 Commits
Author SHA1 Message Date
JaceyShaoandshaojunqi c2dba2dba8 Add H20-3e fused MoE kernel tuning configs for GLM-4.5 (#22433) 
Signed-off-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
Co-authored-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
 2025-08-07 00:24:47 -07:00
Ming YangTyler Michael Smithgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		82216dc21f [Misc] Support routing logic simulation (#21990) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-08-06 23:06:20 -07:00
WeiQing Chenandycyaw66 4be02a3776 [Bugfix] EPLB load statistics problem (#22167) 
Signed-off-by: ycyaw66 <497410282@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
 2025-08-07 04:07:54 +00:00
Syed Muhammad Bin Asif 609b533cb6 [Bugfix] Add proper comparison for package versions (#22314) 
Signed-off-by: Syed Muhammad Bin Asif <syedmba7@connect.hku.hk>
 2025-08-06 20:31:03 -07:00
Wentao Ye eec890c1c1 [Bug] Fix B200 DeepGEMM E8M0 Accuracy Issue (#22399) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-08-06 17:03:53 -07:00
Asaf Joseph Gardinandasafg 46a13949d5 [v1] - Mamba1 Attention Metadata (#21249) 
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-08-06 17:03:42 -07:00
Yongye Zhuandsimon-mo 31f09c615f [gpt-oss] flashinfer mxfp4 (#22339) 
Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
 2025-08-06 12:37:27 -07:00
Jee Jee Li 8e6c7e873f [Bugfix] Fix MoE BNB version (#22260) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-08-05 19:56:22 -07:00
wang.yuqi 586f286789 [Model] Pooling model activation supports per request control by PoolingParams (#20538) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-08-05 00:37:00 -07:00
Wentao Ye d7b28f3415 [Log] DeepGEMM Update Log for Unaligned Problem Size (#22208) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-08-04 19:13:19 -07:00
TJian 6ad6b8e115 [FEAT] Refactor ROPE into module (#22192) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-08-04 19:12:16 -07:00
Po-Han Huang (NVIDIA) bdcb42e45d [NVIDIA] Auto detect modelopt quant and fix DSR1-FP4 weight loading (#22073) 2025-08-04 21:02:55 -04:00
Weixiao Huang c1b4eb048a [feat] move WEIGHT_SCALE_SUPPORTED into raise block to accelerate RLHF weight loading (#21164) 
Signed-off-by: huangweixiao <huangweixiao@msh.team>
 2025-08-04 15:43:06 +08:00
Li, Jiang b5dfb94fa0 [CI/Build][Bugfix] Fix Qwen2.5 tests in CPU CI via fallback silu_and_mul to torch native implementation (#22145) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-08-03 05:34:04 -07:00
Yan Ma 73e1b9b1d4 [xpu]support moe models on XPU platform (#21643) 
Signed-off-by: yan <yan.ma@intel.com>
Signed-off-by: Yan Ma <yan.ma@intel.com>
 2025-08-02 07:49:08 -07:00
Chih-Chieh Yang b690e34824 [Model] Mamba2 preallocate SSM output tensor to avoid d2d copy overhead (#21075) 
Signed-off-by: Chih-Chieh Yang <7364402+cyang49@users.noreply.github.com>
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-08-02 01:59:34 -07:00
Yuxuan ZhangandIsotr0py 25373b6c6c for glm-4.1V update (#22000) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2025-08-02 01:46:57 -07:00
Chih-Chieh Yang c64861d63c [Bugfix] Mamba2 remove bugged initial state condition in chunk scan (#22034) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-08-01 23:55:57 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath a65f46be5e [Misc] DeepGemmExperts : Avoid JIT generation in the hot-path (#21955) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-08-01 19:42:03 -07:00
JartX 3654847db5 feat: Add Support GPTQ Quantization MOE on ROCM vllm serve (#21733) 2025-08-01 21:12:19 -04:00
Varun Sundar RabindranathandVarun Sundar Rabindranath ac45c44d98 [Bugfix] [Performance] DeepEPHighThroughput + DeepSeek : Quant before Dispatch (#21837) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-08-01 10:14:38 -07:00
Wentao Ye c3e0e9337e [Feature] Add Flashinfer MoE Support for Compressed Tensor NVFP4 (#21639) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-31 15:26:11 -07:00
amirkl94andmgoin 207b750e19 [NVIDIA] Add SM100 Flashinfer MoE per tensor scale fp8 backend (#21458) 
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-31 06:00:01 -07:00
Jee Jee Li 0f7919fca0 [Misc] Expand SUPPORTED_HIDDEN_SIZES for DeepEP low-latency kernels (#21818) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-30 20:41:12 -07:00
Po-Han Huang (NVIDIA) ff08e51940 [NVIDIA] Fix Llama4 Scout FP4 functionality issues (#21499) 
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
 2025-07-30 07:33:40 -07:00
Wenhua Cheng ad341c5194 [Bugfix]fix mixed bits and visual language model quantization in AutoRound (#21802) 
Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>
 2025-07-29 07:26:31 -07:00
Isotr0py a4528f0cac [Model]: Fused MoE for nomic-embed-text-v2-moe (#18321) 
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-07-29 03:13:27 -07:00
Wentao Ye 48b763d6b5 [Refactor] Merge Compressed Tensor FP8 CompressedTensorsW8A8Fp8MoEMethod and CompressedTensorsW8A8Fp8MoECutlassMethod (#21775) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-28 19:47:21 -06:00
Nikhil Guptaandmgoin 89ac266b26 [Feat]: Add support for Dynamic Quant 4 bit CPU kleidiai kernels (#17112) 
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-07-28 20:55:15 +00:00
rasmith b361f14e39 [AMD][BugFix] Fix omission of wvSplitK kernel for small batch sizes (1-4) due to torch.compile (#21350) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2025-07-28 15:38:20 -04:00
Asaf Joseph Gardinandasafg a6c050286a [v1][mamba] Added mamba_type into MambaSpec (#21715) 
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
 2025-07-28 08:15:55 +00:00
TJian e626d286f5 [FEAT] [ROCm] [AITER]: Add AITER HIP block quant kernel (#21242) 2025-07-28 05:07:06 +00:00
Jee Jee Li 04ff4be310 [Misc] Add fused_moe configs for Qwen3-Coder-480B-A35B-Instruct-FP8 (#21700) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-07-27 20:12:18 -07:00
Caleb_Du 57c22e57f9 Fix CUDA permute/unpermute for use with DeepGemm Moe (#17934) 
Signed-off-by: Caleb_Du <Caleb_Du@zju.edu.cn>
 2025-07-27 07:08:00 -07:00
Wentao Ye bda9d0535f [Refactor] Refactor MOE NVFP4 Code Base: ModelOpt + Compressed Tensor (#21631) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-27 05:25:21 -07:00
Kaixi Hou de509ae8eb [NVIDIA] Explicitly disable shuffled weights for flashinfer blockscale moe fp8 kernels (#21411) 
Signed-off-by: kaixih <kaixih@nvidia.com>
 2025-07-26 07:10:36 -07:00
Wentao Ye 56e544f24b [Refactor] Remove moe_align_block_size_triton (#21335) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-26 07:08:29 -07:00
Alex Kogan 7ae75fa6d0 [Feature] Add support for MoE models in the calibration-free RTN-based quantization (#20766) 
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
 2025-07-25 18:09:34 -07:00
Wentao Ye 75d29cf4e1 [Perf] Cuda Kernel for Int8 Per Token Group Quant (#21476) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-25 17:07:07 -07:00
Chih-Chieh YangandYu Chin Fabian Lim eab2f3980c [Model] Replace Mamba2 RMSNorm Gated with Fused Triton Kernel (#20839) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Signed-off-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
Signed-off-by: Chih-Chieh Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: Yu Chin Fabian Lim <fabian.lim@gmail.com>
 2025-07-25 06:49:36 -07:00
Cyrus Leung 46d81d6951 [V1] Get supported tasks from model runner instead of model config (#21585) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-25 05:36:45 -07:00
Xu Wenqing 8ed01e32f7 Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-Instruct (#21598) 
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
 2025-07-25 02:36:55 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 2212cd6cfb [Bugfix] DeepGemm utils : Fix hardcoded type-cast (#21517) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-07-24 20:17:29 -07:00
Burkhard RingleinandThomas Parnell ce3a9b1378 [Kernel] adding fused_moe configs for upcoming granite4 (#21332) 
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-07-24 20:16:59 -07:00
Wentao Ye 633f6e804b [Bug] Fix DeepGemm Init Error (#21554) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-24 20:07:22 -07:00
Woosuk Kwon fe56180c7f [MoE] More balanced expert sharding (#21497) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-07-24 15:56:08 -07:00
Shu Wang 1b25f1fe75 Update flashinfer CUTLASS MoE Kernel (#21408) 
Signed-off-by: Shu Wang. <shuw@nvidia.com>
 2025-07-24 08:13:31 -07:00
Nick Hill f0f4de8f26 [Misc] Fix duplicate FusedMoEConfig debug messages (#21455) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-24 01:27:30 -07:00
Chengji YaoandSimon Mo e74bfc70e4 [TPU][Bugfix] fix moe layer (#21340) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-07-24 00:38:39 -07:00
Michael Goin f002e9a870 [Cleanup] Only log MoE DP setup warning if DP is enabled (#21315) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-23 00:02:48 -07:00