iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
15,230 Commits 3 Branches 0 Tags
d215d1efca7a18eb2a19007f229bbb070bfbee93
Commit Graph
940 Commits
Author SHA1 Message Date
Kunshang Ji 14771f7150 [XPU] support MLA model on Intel GPU (#37143) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-25 17:43:42 +08:00
Andreas Karatzas 679c6a3ecc [Bugfix][ROCm][MoE] Fix mxfp4 oracle regressions from #37128 (#37787) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-03-25 08:17:33 +08:00
Li, Jiang 352b90c4a4 [Bugfix] Add replacement of _compute_slot_mapping_kernel on CPU (#37987) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-03-24 07:00:20 -07:00
RanranandWentao Ye dc6908ac6a [Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007) 
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-03-23 18:31:14 -04:00
Kyle Sayers 38364a7e32 [Sparse24] [Deprecation] Remove Sparse24 CT integration and kernels (#36799) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-03-23 16:03:29 -04:00
Kunshang Ji debd6e768c [XPU][MoE Refactor] Refactor xpu mxfp4 support into oracle (#37784) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-23 11:10:41 +00:00
Kunshang Ji 27d5ee3e6f [FP8]add FP8 WoQ kernel abstraction. (#32929) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
 2026-03-23 09:47:47 +00:00
Chuan (Richard) Li e99fb98867 [ROCm] Fix fused_moe_fake signature mismatch and other AITER bugs (#36100) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-23 15:48:31 +08:00
Matthias GehreandMichael Goin 410d300893 [ROCm][Refactor] Enable AWQMarlinConfig on ROCm to use choose_mp_linear_kernel (#36505) 
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-23 15:36:08 +08:00
Yongye Zhu c058ff44d4 [Bigfix]fix lora test by pass padded size back to the layer (#37811) 2026-03-22 13:20:13 -06:00
Wentao Ye 77d24c4bfe [Bug] Fix fp8 deepgemm batch invariant (#37718) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-03-22 08:57:20 -04:00
Robert ShawandRobert Shaw eeee5b262d [Quantization][Deprecation] Remove PTPC FP8 (#32700) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-03-21 22:10:16 +00:00
Robert Shaw 5ad0446572 Revert "Consolidate AWQ quantization into single awq_marlin.py file" (#37768) 2026-03-21 17:20:41 -04:00
Robert ShawandClaude 8cc700dd6a Consolidate AWQ quantization into single awq_marlin.py file 
Merge awq.py and awq_marlin.py into a single file, eliminating the
circular import between them. awq.py becomes a backward-compat shim.
Follows the same structure as gptq_marlin.py.

Co-authored-by: Claude

Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-03-21 17:09:17 -04:00
Yongye ZhuandClaude Opus 4.6 87bd91892f [MoE Refactor] Mxfp4 oracle rebased (#37128) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 03:37:04 +00:00
xuebwang-amd 44eea10f68 [ROCm][Quantization] make quark ocp mx dtype parser robust for weight-only quantization (#36232) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
 2026-03-20 10:10:03 -05:00
Michael Goin 9482b0b085 [Bugfix] Remove assertion for NVFP4 scale dynamic range (#37465) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2026-03-18 15:37:49 -07:00
elvischenv 296839a1b0 [Perf] Eliminate padding and slicing op for GPT-OSS with Flashinfer MXFP4 MXFP8 MoE (#30647) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2026-03-18 15:01:26 +00:00
Chao-Ju ChenandMichael Goin 245758992e [Bugfix] Rescale NVFP4 weight scales to fix BF16 dequant underflow (#34577) 
Signed-off-by: ricky-chaoju <ricky.chen@infinirc.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-03-17 20:48:42 +00:00
Andrey Talman 68f783a727 [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) 
Signed-off-by: atalman <atalman@fb.com>
 2026-03-17 18:47:59 +00:00
xiao-llmYanwen LinKim Hee Su
 		24b4272a8c Fix infinite recursive search issue in quark.py (#32779) 
Signed-off-by: Yanwen Lin <lyw1124278064@gmail.com>
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com>
Signed-off-by: kimheesu <wlskaka4@gmail.com>
Co-authored-by: Yanwen Lin <lyw1124278064@gmail.com>
Co-authored-by: Kim Hee Su <wlskaka4@gmail.com>
 2026-03-17 07:19:15 +00:00
EdalatiAli e5b807607c [Quant][Feature] Support online MXFP8 quantization for MoE and dense models (#35448) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
 2026-03-16 18:07:39 -04:00
Elvir CrnčevićTyler Michael SmithClaude Opus 4.6
 		fd4d96302a Fix eplb nvfp4 experts hook (#37217) 
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: Elvir Crncevic <elvir@anthropic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-16 22:03:54 +00:00
Divakar Verma 6341d43043 [ROCm][Quantization] add quark w4a8 mxfp4_fp8 for LinearLayer (#35316) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-03-13 19:44:24 +00:00
Itay AlroyYongji WuRon Tourgeman
 		d5af196c18 [2/N] Elastic EP Milestone 2: Integrating NIXL-EP (#35627) 
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
Co-authored-by: Yongji Wu <wuyongji317@gmail.com>
Co-authored-by: Ron Tourgeman <rtourgeman@nvidia.com>
 2026-03-13 09:25:33 -04:00
Jaewon aaa3092f51 [MoE] Add routing simulation override for MXFP4 quantized MoE (#33595) 
Signed-off-by: Jaewon Lee <jaewon@meta.com>
 2026-03-13 00:30:44 +00:00
Michael GoinandClaude 57431d8231 [UX] Only show FP4 Marlin fallback warning for w4a4 models (#36806) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-03-12 05:19:35 -04:00
Hongbin Guo 4bf533623b [Doc] Fix duplicate words in comments (#36713) 
Signed-off-by: Hongbin10 <jdmjdm1998@163.com>
 2026-03-10 21:28:31 -07:00
danisereb 0a6a3a1290 Add support for ModelOpt MXFP8 MoE models (#35986) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-03-08 13:00:05 -07:00
Jiayi YanandHarry Mellor 6a895197fa [Bugfix][CI] fix typos (#34934) 
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-05 17:05:46 +00:00
Kunshang JiandHarry Mellor 16d2ad1d38 [Hardware] Replace torch.cuda.empty_cache with torch.accelerator.empty_cache (#30681) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-04 09:49:47 +00:00
Chuan (Richard) Li 5dc3538736 [ROCm][Bugfix] Fall back from CK MXFP4 MoE when GEMM dimensions are unsupported (#35893) 
Signed-off-by: Li <chuali@amd.com>
 2026-03-04 08:30:54 +00:00
bnellnmandRobert Shaw a9b8b13e5c [Bugfix] Fix misnamed parameter in compressed_tensors_moe.py (#35813) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-03-03 16:29:57 -05:00
Robert ShawRobert ShawRobert Shaw
 		97995f6376 [MoE Refactor] Create MK for TRTLLM Kernels (#32564) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
 2026-03-03 10:39:50 -08:00
Kunshang Ji 92f5d0f070 [XPU] fix mxfp4 activation type (#35691) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-02 11:48:39 +08:00
Seungho Yoon 5a435507d8 fix(mxfp4): return is_monolithic=False when LoRA is enabled for Triton backend (#35382) 
Signed-off-by: Seungho Yoon <yoonsnowdev@gmail.com>
 2026-03-01 09:59:30 -05:00
Douglas LehrDoug LehrCursorRohan Potdar
 		d5b6f3ba36 [ROCm][Quantization] Add Composable Kernel (CK) backend support for M… (#34301) 
Signed-off-by: Doug Lehr <douglehr@amd.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
 2026-02-28 03:37:01 +00:00
Pavani Majety 6283021142 [Bugfix] Fix KV Scale loading for MLA Models (#35430) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2026-02-26 23:38:19 +00:00
Aleksandr MalyshevandAleksandr Malyshev 01923eec70 [ROCm][Quantization] GPT OSS Upstream MoE wmxfp4_afp8 with static scales (#30357) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
 2026-02-26 16:50:16 -06:00
Tyler Michael SmithandClaude Opus 4.6 eb19955c37 [WideEP] Remove pplx all2all backend (#33724) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-26 14:30:10 -08:00
sychen52 d0105b84f0 add mixed precision support for modelopt (#35047) 
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
 2026-02-26 21:56:24 +00:00
Douglas LehrDoug LehrRohan PotdarGregory Shtrasberg
 		ec8ab9d254 [ROCm] Add dynamic mxfp4 quantization for DeepSeek V2 projection layers (#34157) 
Signed-off-by: Doug Lehr <douglehr@amd.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: Doug Lehr <douglehr@amd.com>
Co-authored-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
 2026-02-26 10:00:49 -06:00
Wentao Ye 05972ea7e5 [Refactor] Remove dead or duplicate func utils or variables (#35318) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-02-26 10:57:56 -05:00
Yongye Zhu 1976356ee6 [MoE Refactor] MXFP4 Cutlass Experts to MK (#34542) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-02-25 17:32:39 -08:00
Kunshang Ji 8ad54a991b [Platform] Add current_platform.num_compute_units interface (#35042) 
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
 2026-02-24 22:22:49 -08:00
danisereb 9609b1f18d Integrate flashinfer mm_mxfp8 in ModelOpt MXFP8 (#35053) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-02-24 08:45:13 -07:00
danisereb a0c7081695 Fix fallback to default tactic (flashinfer autotuner) with trtllm_fp4_block_scale_moe (#35088) 
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
 2026-02-24 07:25:44 -08:00
Eldar KurtićEldar KurticYour Name
 		a87cc50859 [Attn,KV-cache] Use per-head scales in the attention selector (#34281) 
Signed-off-by: Your Name <you@example.com>
Signed-off-by: Eldar Kurtic <research@neuralmagic.com>
Co-authored-by: Eldar Kurtic <research@neuralmagic.com>
Co-authored-by: Your Name <you@example.com>
 2026-02-24 09:02:43 -05:00
BadrBasowidvllmellmTJian
 		6af03f2394 [Refactor] [1/N] Reorganize kernel abstraction directory (#34055) 
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-02-24 06:47:22 +00:00
Jia GuoandClaude Opus 4.6 ec85340531 [Quantization] Support FP8 MoE bias for models like GPT-OSS (#34906) 
Signed-off-by: jasperjiaguo <jasperg662@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-02-23 19:07:47 -08:00