|
5bb8d2767a
|
[Kernel] Batch invariant NVFP4 linear using cutlass (#39912)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-23 09:41:12 -04:00 |
|
|
Ma JianandKunshang Ji
|
a5bbd81e2e
|
[XPU]feat: enable FP8 block-scaled quantization on XPU (#42952)
Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-05-23 12:33:18 +08:00 |
|
|
Michael Goin
|
9b54e50e2c
|
[Deprecation] Mark env vars covered by --moe-backend / --linear-backend (#43148)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
|
2026-05-21 12:51:12 -07:00 |
|
|
Wentao YeandIsotr0py
|
68e07d5916
|
[Bug] Fix ci issue assert output_size is not None AssertionError (#43261)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-05-21 16:58:09 +08:00 |
|
|
Flora Feng
|
2a43b407c5
|
[Bugfix][CI] Add missing import of pad_nvfp4_activation_for_cutlass in flashinfer (#43237)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-05-20 11:59:12 -07:00 |
|
|
Wentao YeandMatthew Bonanni
|
53ff50fcd3
|
[Perf] Optimize CutlassFP8ScaledMMLinearKernel when padding needed by pre-weight processing, 13.5% TTFT improvement (#42651)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-20 11:57:42 -07:00 |
|
|
meena-at-workandClaude Sonnet 4.6
|
363fc84407
|
Integrate flashinfer b12x MoE and FP4 GEMM kernels for SM120/121 (#40082)
Signed-off-by: Meenakshi Venkataraman <meenakshiv@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-05-20 17:21:11 +00:00 |
|
|
Kunshang Ji
|
36dcaf25d8
|
[XPU] add gptq(int4) support (#37844)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-05-19 11:17:09 +08:00 |
|
|
Wentao Ye
|
37ece593c1
|
[Perf] Padded nvfp4 quant kernel to remove additional copy, 2.4%~5.7% e2e performance improvement (#42774)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-18 16:38:12 -07:00 |
|
|
zofiaandKunshang Ji
|
0fa888465e
|
[XPU] fix weight scale shape (#42725)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-05-17 16:55:10 +08:00 |
|
|
Michael Goin
|
b2a27b82d9
|
[Kernel][UX] Add --linear-backend arg for linear kernel selection (#39538)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-05-15 17:07:39 -07:00 |
|
|
Libin Tangandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
9946c38b7f
|
[XPU] Fix double-transpose in XPUFP8ScaledMMLinearKernel for W8A8 quant method (#41689)
Signed-off-by: Libin Tang <libin.tang@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-14 17:17:39 +08:00 |
|
|
TJianandYongye Zhu
|
a8887c208f
|
[Bugfix] [ROCm] [DSV4] [Perf] Add aiter mhc support (#41946)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-05-13 21:43:15 +08:00 |
|
|
zofiaandKunshang Ji
|
0ddaf6dffa
|
[XPU] [CT] Enable CT W4A4MxFp4 path and add xpu kernel (#38896)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-05-13 06:43:00 -07:00 |
|
|
Dipika Sikka
|
a7b801e26d
|
[MXFP4] Support for linear layers + compressed-tensors integration (#41664)
|
2026-05-12 07:49:33 -04:00 |
|
|
John Calderon
|
8a2fc80b84
|
[CUDA][CUTLASS] Enable cutlass scaled mm for non-compatible sizes (#41868)
Signed-off-by: John Calderon <jcalderon@nvidia.com>
|
2026-05-08 15:58:05 -07:00 |
|
|
Yuwen Zhou
|
809b98e5b7
|
[CPU] Add FP8 W8A16 linear support (#41186)
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
|
2026-05-06 07:05:27 +00:00 |
|
|
|
628c436301
|
[New Model][ROCm] Add AMD support for DeepSeek V4 (#40871)
Signed-off-by: ganyi <ygan@amd.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com>
Co-authored-by: ganyi <ygan@amd.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaavllm <tunjian.tan@amd.com>
|
2026-05-05 08:55:37 -07:00 |
|
|
Andreas Karatzas
|
5e2c37facd
|
[ROCm][CI] Add missing quantization methods and fix online quant test failures (#39801)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-04-27 15:08:57 -05:00 |
|
|
+7
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
|
Hashem Hashemi
|
3a30eaa1d7
|
Properly enable wvSplitK fp8 path for RDNA (#37712)
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
|
2026-04-20 10:09:24 -05:00 |
|
|
nemanjaudovic
|
58631d7c3f
|
[Bugfix] Fix scaled_mm output narrowing for 3D input tensors (#38093)
Signed-off-by: nemanjaudovic <nudovic@amd.com>
|
2026-04-20 16:58:39 +08:00 |
|
|
Maral
|
c0c98b8b9a
|
[Bugfix] Add Marlin kernel in block scaled mm kernel selection. (#40105)
Signed-off-by: maral <maralbahari.98@gmail.com>
|
2026-04-17 10:20:32 +00:00 |
|
|
vllmellm
|
5f7fab881a
|
[ROCm][FEAT] Integrate aiter gemm w8a8 ptpc (#33773)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
|
2026-04-16 09:55:29 +08:00 |
|
|
Wentao Ye
|
2ad1029233
|
[Bug] Fix batch invariance nvfp4 support (#39820)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-04-14 17:08:17 -04:00 |
|
|
JartX
|
f414f90601
|
[Bugfix][Kernel][ROCm] Fix triton_w4a16 scales mismatch when BLOCK_K > group_size (#39705)
Signed-off-by: JartX <sagformas@epdcenter.es>
|
2026-04-13 14:29:45 -04:00 |
|
|
zofia
|
1b19bd7589
|
[MXFP8] [XPU] add a new compressed tensor schema and add a xpu mxfp8 gemm kernel (#38707)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
|
2026-04-13 16:59:20 +08:00 |
|
|
Yan Ma
|
394ff86965
|
[XPU][CT] support per-channel quantization in xpu fp8 linear method (#38316)
Signed-off-by: Yan Ma <yan.ma@intel.com>
|
2026-04-12 02:46:28 +00:00 |
|
|
Michael Goin
|
11e2375fe2
|
[Refactor] Move MXFP8 GEMM management into MxFp8LinearKernel (#39205)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-04-10 14:02:03 -07:00 |
|
|
|
d468322dc1
|
[Kernel][Hardware][AMD] Add TritonW4A16LinearKernel for ROCm (#37352)
Signed-off-by: jatseng-ai <jatseng@amd.com>
Signed-off-by: jatseng-ai <janet.tseng@amd.com>
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-04-10 10:25:27 +00:00 |
|
|
Kunshang Ji
|
55d037e2e5
|
[CT][FP8][Marlin] refactor CompressedTensorsW8A16Fp8 to use kernel abstraction (#38244)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
|
2026-04-10 09:58:35 +08:00 |
|
|
Michael Goin
|
2800706f06
|
[Refactor] Move NVFP4 GEMM management into NvFp4LinearKernel (#39129)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-04-09 15:05:36 -04:00 |
|
|
Maral
|
2e9034c998
|
[W8A8 Block Linear Refactor][2/N] Remove W8A8Fp8BlockLinearOp and adopt Fp8 block linear kernel selections. (#33892)
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: Maral <maralbahari.98@gmail.com>
|
2026-04-09 08:50:39 +08:00 |
|
|
Andreas Karatzas
|
780ba37458
|
[ROCm][Quantization] Add asymmetric INT8 quantization support to TritonInt8ScaledMMLinearKernel (#38501)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-04-06 09:42:10 +08:00 |
|
|
zhangyiming
|
1ac6694297
|
[OOT] Add OOT support for linear kernel. (#37989)
Signed-off-by: menogrey <1299267905@qq.com>
|
2026-03-31 14:33:21 +08:00 |
|
|
Li, Jiang
|
becaed6ec8
|
[CPU] Support CT W4A16 on CPU MP kernel (#38219)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-03-27 14:15:28 +08:00 |
|
|
Andreas Karatzas
|
a8e48a7b85
|
[CI] Fix conch kernel crash on 3D input by reshaping to 2D before GEMM (#38178)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-03-26 11:46:03 -05:00 |
|
|
BadrBasowidandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
e6bf9f15ec
|
[Bugfix][CI] Fix Marlin FP8 Linear Kernel for Compressed Tensors Format (#38092)
Signed-off-by: BadrBasowid <Badr.Basowid@gmail.com>
Signed-off-by: BadrBasowid <61441185+BadrBasowid@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-03-25 21:11:43 -07:00 |
|
|
RanranandWentao Ye
|
dc6908ac6a
|
[Bugfix] Register VLLM_BATCH_INVARIANT in envs.py to fix spurious unknown env var warning (#35007)
Signed-off-by: Ranran <1012869439@qq.com>
Signed-off-by: Ranran <hzz5361@psu.edu>
Signed-off-by: ran <hzz5361@psu.edu>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-03-23 18:31:14 -04:00 |
|
|
Kunshang Ji
|
27d5ee3e6f
|
[FP8]add FP8 WoQ kernel abstraction. (#32929)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
|
2026-03-23 09:47:47 +00:00 |
|
|
Matthias GehreandMichael Goin
|
410d300893
|
[ROCm][Refactor] Enable AWQMarlinConfig on ROCm to use choose_mp_linear_kernel (#36505)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-03-23 15:36:08 +08:00 |
|
|
Matthias Gehre
|
5e806bcf54
|
[Bugfix] Fix ConchLinearKernel channelwise quantization (group_size=-1) (#37329)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-03-20 10:32:21 -05:00 |
|
|
Matthias Gehre
|
56a62c310c
|
[Bugfix] Reject channelwise quantization (group_size <= 0) in ExllamaLinearKernel (#37331)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-03-20 10:31:57 -05:00 |
|
|
Tianmu Li
|
47b7af0d87
|
[Feat] Enable CompressedTensorW4A8Int for XPU (#37207)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
|
2026-03-20 02:34:28 +00:00 |
|
|
Andrey Talman
|
68f783a727
|
[Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673)
Signed-off-by: atalman <atalman@fb.com>
|
2026-03-17 18:47:59 +00:00 |
|
|
Martin Hickey
|
700a1ddc65
|
[Misc] Use envs module to get VLLM_DISABLED_KERNELS (#35776)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-03-11 13:37:46 +00:00 |
|
|
Jiayi YanandHarry Mellor
|
6a895197fa
|
[Bugfix][CI] fix typos (#34934)
Signed-off-by: 1195343015 <1195343015@qq.com>
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-03-05 17:05:46 +00:00 |
|
|
Kunshang Ji
|
66a2209645
|
[Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize (#36085)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-03-05 10:36:39 +00:00 |
|
|
Fadi ArafehandTyler Michael Smith
|
4171ff6dd9
|
[CPU][Feat] Enable KleidiAI INT8_W4A8 for all input dtypes (#34890)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-02-26 05:00:10 +00:00 |
|
|
Kunshang Ji
|
8ad54a991b
|
[Platform] Add current_platform.num_compute_units interface (#35042)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
|
2026-02-24 22:22:49 -08:00 |
|