iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15,396 Commits 3 Branches 0 Tags
ea7bfde6e40d1c0bd2b03b2ace3ded561dff7026
Commit Graph
956 Commits
Author SHA1 Message Date
Yintong Lu f09daea261 [CPU] Support int8 compute mode in CPU AWQ (#35697) 
Signed-off-by: Yintong Lu <yintong.lu@intel.com>
 2026-03-31 15:27:37 +08:00
Johnny b4a2f3ac36 [NVIDIA] Bugfix NVFP4 DGX Spark and RTX50 (#38423) 
Signed-off-by: johnnynunez <johnnynuca14@gmail.com>
Signed-off-by: Johnny <johnnynuca14@gmail.com>
 2026-03-30 09:36:18 -07:00
Hongxia YangandHongxia Yang dbdd9ae067 [ROCm][Bugfix] fix exception related to trust_remote_code for MiniMax-M2.1-MXFP4 (#37698) 
Signed-off-by: Hongxia Yang <hongxiay.yang@amd.com>
Co-authored-by: Hongxia Yang <hongxiay.yang@amd.com>
 2026-03-30 15:49:23 +00:00
Matthias GehreandMichael Goin e8b055a5ac [Bugfix] Handle ParallelLMHead in compressed-tensors get_quant_method (#37291) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-30 07:30:52 -07:00
PikaPikachu 63babd17f1 [Model][Quantization] Add GGUF support for MiniMax-M2.1 (#36965) 
Signed-off-by: kangletian <Letian.Kang@amd.com>
 2026-03-30 14:24:06 +08:00
Kyle Sayers d28d86e8a3 [QeRL] Fix online quantized reloading (#38442) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-29 14:56:41 -06:00
IriKa 148a5c1226 [Bugfix]fix output Nan/Inf in marlin if dtype=float16 (#33972) 
Signed-off-by: IriKa Qiu <qiujie.jq@gmail.com>
 2026-03-27 16:36:08 -07:00
Kyle Sayers 648edcf729 [QeRL] Compose online quantization with quantized reloading (#38032) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-27 13:22:33 -07:00
Gregory Shtrasberg b8665383df [ROCm] Fix GPT-OSS import for triton 3.6 (#37453) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2026-03-27 18:00:57 +00:00
Rohan PotdarandAndreas Karatzas 0e9358c11d {ROCm]: gpt-oss fusion/padding fixes (#38043) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-27 12:19:15 -04:00
Bowen Bao 0ae89f18fd [Refactor] Move FusedMoE hidden_size roundup to quant_method (#34285) 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
 2026-03-26 23:38:26 -07:00
Zhewen LiandZhewen Li be1a85b7a2 Revert "[MoE Kernel] Flashinfer nvfp4 cutedsl moe kernel integration" (#38050) (#38169) 
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-03-26 07:59:09 -07:00
Vadim Gimpelson 52069012fe [Bugfix] Fix DeepGemm E8M0 accuracy degradation for Qwen3.5 FP8 on Blackwell (#38083) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2026-03-26 01:21:47 -07:00
BadrBasowidandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> e6bf9f15ec [Bugfix][CI] Fix Marlin FP8 Linear Kernel for Compressed Tensors Format (#38092) 
Signed-off-by: BadrBasowid <Badr.Basowid@gmail.com>
Signed-off-by: BadrBasowid <61441185+BadrBasowid@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-03-25 21:11:43 -07:00
Andreas KaratzasWoosuk KwonMatthew Wong
 		7d6917bef5 [ROCm] Fix MoE kernel test failures on gfx950 (#37833) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
 2026-03-25 13:46:40 -05:00
Yongye Zhu 678b3c99e8 [MoE Kernel] Flashinfer nvfp4 cutedsl moe kernel integration (#38050) 2026-03-25 10:16:40 -07:00
Kunshang Ji 14771f7150 [XPU] support MLA model on Intel GPU (#37143) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-25 17:43:42 +08:00
Andreas Karatzas 679c6a3ecc [Bugfix][ROCm][MoE] Fix mxfp4 oracle regressions from #37128 (#37787) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-25 08:17:33 +08:00
Li, Jiang 352b90c4a4 [Bugfix] Add replacement of _compute_slot_mapping_kernel on CPU (#37987) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-24 07:00:20 -07:00
RanranandWentao Ye dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) 
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-23 18:31:14 -04:00
Kyle Sayers 38364a7e32 [Sparse24] [Deprecation] Remove Sparse24 CT integration and kernels (#36799) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-23 16:03:29 -04:00
Kunshang Ji debd6e768c [XPU][MoE Refactor] Refactor xpu mxfp4 support into oracle (#37784) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-23 11:10:41 +00:00
Kunshang Ji 27d5ee3e6f [FP8]add FP8 WoQ kernel abstraction. (#32929) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-03-23 09:47:47 +00:00
Chuan (Richard) Li e99fb98867 [ROCm] Fix fused_moe_fake signature mismatch and other AITER bugs (#36100) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-23 15:48:31 +08:00
Matthias GehreandMichael Goin 410d300893 [ROCm][Refactor] Enable AWQMarlinConfig on ROCm to use choose_mp_linear_kernel (#36505) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-23 15:36:08 +08:00
Yongye Zhu c058ff44d4 [Bigfix]fix lora test by pass padded size back to the layer (#37811) 2026-03-22 13:20:13 -06:00
Wentao Ye 77d24c4bfe [Bug] Fix fp8 deepgemm batch invariant (#37718) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-22 08:57:20 -04:00
Robert ShawandRobert Shaw eeee5b262d [Quantization][Deprecation] Remove PTPC FP8 (#32700) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-21 22:10:16 +00:00
Robert Shaw 5ad0446572 Revert "Consolidate AWQ quantization into single awq_marlin.py file" (#37768) 2026-03-21 17:20:41 -04:00
Robert ShawandClaude 8cc700dd6a Consolidate AWQ quantization into single awq_marlin.py file 
Merge awq.py and awq_marlin.py into a single file, eliminating the
circular import between them. awq.py becomes a backward-compat shim.
Follows the same structure as gptq_marlin.py.

Co-authored-by: Claude

Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 17:09:17 -04:00
Yongye ZhuandClaude Opus 4.6 87bd91892f [MoE Refactor] Mxfp4 oracle rebased (#37128) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 03:37:04 +00:00
xuebwang-amd 44eea10f68 [ROCm][Quantization] make quark ocp mx dtype parser robust for weight-only quantization (#36232) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-03-20 10:10:03 -05:00
Michael Goin 9482b0b085 [Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-03-18 15:37:49 -07:00
elvischenv 296839a1b0 [Perf] Eliminate padding and slicing op for GPT-OSS with Flashinfer MXFP4 MXFP8 MoE (#30647) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2026-03-18 15:01:26 +00:00
Chao-Ju ChenandMichael Goin 245758992e [Bugfix] Rescale NVFP4 weight scales to fix BF16 dequant underflow (#34577) 
Signed-off-by: ricky-chaoju <ricky.chen@infinirc.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-17 20:48:42 +00:00
Andrey Talman 68f783a727 [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) 
Signed-off-by: atalman <atalman@fb.com>
 2026-03-17 18:47:59 +00:00
xiao-llmYanwen LinKim Hee Su
 		24b4272a8c Fix infinite recursive search issue in quark.py (#32779) 
Signed-off-by: Yanwen Lin <lyw1124278064@gmail.com>
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Signed-off-by: kimheesu <wlskaka4@gmail.com>
Co-authored-by: Yanwen Lin <lyw1124278064@gmail.com>
Co-authored-by: Kim Hee Su <wlskaka4@gmail.com>
 2026-03-17 07:19:15 +00:00
EdalatiAli e5b807607c [Quant][Feature] Support online MXFP8 quantization for MoE and dense models (#35448) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
 2026-03-16 18:07:39 -04:00
Elvir CrnčevićTyler Michael SmithClaude Opus 4.6
 		fd4d96302a Fix eplb nvfp4 experts hook (#37217) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Elvir Crncevic <elvir@anthropic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-16 22:03:54 +00:00
Divakar Verma 6341d43043 [ROCm][Quantization] add quark w4a8 mxfp4_fp8 for LinearLayer (#35316) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-03-13 19:44:24 +00:00
Itay AlroyYongji WuRon Tourgeman
 		d5af196c18 [2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-03-13 09:25:33 -04:00
Jaewon aaa3092f51 [MoE] Add routing simulation override for MXFP4 quantized MoE (#33595) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
 2026-03-13 00:30:44 +00:00
Michael GoinandClaude 57431d8231 [UX] Only show FP4 Marlin fallback warning for w4a4 models (#36806) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-03-12 05:19:35 -04:00
Hongbin Guo 4bf533623b [Doc] Fix duplicate words in comments (#36713) 
Signed-off-by: Hongbin10 <jdmjdm1998@163.com>
 2026-03-10 21:28:31 -07:00
danisereb 0a6a3a1290 Add support for ModelOpt MXFP8 MoE models (#35986) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-03-08 13:00:05 -07:00
Jiayi YanandHarry Mellor 6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Kunshang JiandHarry Mellor 16d2ad1d38 [Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 09:49:47 +00:00
Chuan (Richard) Li 5dc3538736 [ROCm][Bugfix] Fall back from CK MXFP4 MoE when GEMM dimensions are unsupported (#35893) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-04 08:30:54 +00:00
bnellnmandRobert Shaw a9b8b13e5c [Bugfix] Fix misnamed parameter in compressed_tensors_moe.py (#35813) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-03-03 16:29:57 -05:00
Robert ShawRobert ShawRobert Shaw
 		97995f6376 [MoE Refactor] Create MK for TRTLLM Kernels (#32564) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
 2026-03-03 10:39:50 -08:00