iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
9,818 Commits 3 Branches 0 Tags
c8bde93367fb252eca1e9a6ae78650caa4a9a951
Commit Graph
1204 Commits
Author SHA1 Message Date
Chenxi YangandChenxi Yang 0d235b874a Add CUTLASS FP8 MOE benchmark scripts and kernel config (#25302) 
Signed-off-by: Chenxi Yang <cxyang@fb.com>
Co-authored-by: Chenxi Yang <cxyang@fb.com>
 2025-09-23 18:07:42 -06:00
Juan Villamizarandjpvillam bde2a1a8a4 [ROCm] Small functional changes for gptoss (#25201) 
Signed-off-by: jpvillam <jpvillam@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
 2025-09-23 23:39:50 +00:00
Thomas ParnellandChih-Chieh-Yang 5e25b12236 [Kernel] [Mamba] Remove BLOCK_H=1 from list of tuneable configurations for _chunk_cumsum_fwd_kernel (#25197) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
 2025-09-23 23:23:30 +00:00
Michael Goin 7361ab379f Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-23 22:48:40 +00:00
Gregory Shtrasberg 690f948e4a [Bugfix] Fix for the import error from #24588 (#25481) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-09-23 21:31:08 +00:00
ElizaWszolaLuka GovedičMichael GoinLuka Govedič
 		63400259d0 [Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-09-23 12:03:10 -07:00
Hashem Hashemi a3a7828010 [ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) 
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>
 2025-09-23 14:31:45 -04:00
bnellnmandMichael Goin f11e3c516b [Kernels] Support blocked fp8 quantization for compressed tensors MoE (#25219) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-23 16:11:34 +00:00
Lucas WilkinsonSage Mooreyewentao256Tyler Michael Smith
 		cc1dc7ed6d [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-23 16:02:10 +00:00
Thomas Parnell a903669e10 [V1] Remove V0 code paths for Hybrid models (#25400) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-09-23 08:26:13 -07:00
Joelandyoukaichao 61d1b35561 [BugFix] Register expert_map as named buffer for wake_up and sleep (#25458) 
Signed-off-by: wuxibin <wuxibin@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2025-09-23 21:49:13 +08:00
Ming Yang ba8d2165b6 Handle triton kernel import exception (#25319) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-09-23 00:56:00 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath e8db44f883 [DP/EP][GPTOSS] Use triton matmul-ogs kernels for GPTOSS DP/EP (#24588) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-09-22 21:01:09 -07:00
JJJYmmmandliuye.hj fc97733da8 [feat] Support MRoPE + YaRN (#25384) 
Signed-off-by: liuye.hj <liuye.hj@alibaba-inc.com>
Co-authored-by: liuye.hj <liuye.hj@alibaba-inc.com>
 2025-09-23 03:04:47 +00:00
Bowen Wanggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Tyler Michael Smith
 		06a41334c7 [EPLB] Reduce EPLB Inference Overhead (#24573) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-09-22 16:31:05 +00:00
Sara-KS ac243886b0 [Kernel] MI-300X triton moe configs (#23445) 
Signed-off-by: Sara Kokkila Schumacher <saraks@ibm.com>
 2025-09-22 14:29:54 +00:00
Woosuk Kwon bc6e542d9f Remove V0 attention backends (#25351) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-21 16:03:28 -07:00
Woosuk Kwon 1c3ffdbecc [V0 Deprecation] Remove V0 sampling metadata (#25345) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-21 10:37:11 -07:00
Woosuk Kwon 26e673fe93 [V0 Deprecation] Remove V0 Sequence class & Sampler (#25332) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-21 08:52:15 -07:00
Michael Goin e08a3a3fdb [CI Failure] Disable FlashInfer RoPE to unblock CI (#25299) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-09-20 08:16:56 +00:00
ZhiyuandMichael Goin 431535b522 Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) 
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-09-19 22:40:33 +00:00
David-WenWentao YeMichael Goingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 		ddc9048394 Fix: Correct FusedMoE layer reference in auto_round quantization (#24818) 
Signed-off-by: David-Wen <18927700430@163.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-19 20:44:24 +00:00
Michael GoinandLuka Govedič 48ecb4438b [Perf] Use FlashInfer RoPE for RotaryEmbedding.forward_cuda when available (#21126) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-09-19 14:06:49 -06:00
bnellnm 4bdf400218 [Bugfix] Fix chunked a2_scales in modular kernels (#25264) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-19 19:42:01 +00:00
Varun Sundar RabindranathandVarun Sundar Rabindranath 7852b82b93 [Bugfix] GPT OSS Attritbute error on H100 (#25228) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-09-19 13:14:09 -06:00
samzong 138f0d1e75 [Docs] add __init__.py to vllm/model_executor/layers/quantization/compressed_tensors/transform (#24974) 
Signed-off-by: samzong <samzong.lu@gmail.com>
 2025-09-19 18:32:27 +00:00
Isotr0py cea91a32f2 [Kernel][Performance] Add Triton kernel for Qwen3-VL interleaved MRoPE (#25055) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-19 10:27:49 +00:00
Li, Jiang 825fdb11ad [Bugfix][CPU] Add placeholder to avoid import errors when using fused_moe ops on platforms without triton (#25137) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-19 07:41:12 +00:00
Li, Jiang 8c1d4acbfe [CPU] Disable oneDNN linear on non-x86 platforms (#25166) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-09-19 07:27:22 +00:00
Vadim Gimpelson 072d7e53e5 [PERF] Add conv1d metadata to GDN attn (#25105) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-09-18 14:27:49 +00:00
Michael Goin fbd6523ac0 Refactor dense FP8 tensor/channel/block utils and add CT FP8 block (#21404) 2025-09-18 08:53:45 -04:00
Asaf Joseph Gardin 66072b36db [Bugfix][Mamba] - Fix Conv State Kernel FP32 Support (#24883) 
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
 2025-09-18 12:21:17 +00:00
Jee Jee Li 37970105fe [Model] Improve Pooling Model (#25149) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-18 11:04:21 +00:00
PunitvaraandHarry Mellor 05b044e698 [Doc] Fix cross-reference warnings (#25058) 
Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-18 02:05:16 -07:00
YiwenC 9d8a2d86d2 [EPLB] Add EPLB support for hunyuan_v1 (#23078) 2025-09-18 04:51:35 +00:00
bnellnm dc2979c585 [Kernels] Overlap shared experts with combine instead of dispatch (#24254) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-18 12:10:21 +08:00
bnellnm 5963b98b46 [Kernel] Delegate construction of FusedMoEQuantConfig to FusedMoEMethodBase subclasses (#22537) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-09-17 17:43:31 -06:00
Tao He dd6a910aac [Bugfix][Qwen3-Next] fixes the varlen issue in qwen3-next's MTP implementation. (#24957) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-09-17 21:59:09 +08:00
haoyangli-amdandHaoyang Li ca2d1925ef [Rocm] [quantization] Fix quark ptpc moe and add test case (#24649) 
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
Co-authored-by: Haoyang Li <haoyang.li@amd.com>
 2025-09-16 22:15:13 -07:00
Roger WangHuang Jie松灵Isotr0py
 		0f7acdd73c [Model] Support Qwen3-VL Model Series (#24727) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Huang Jie <92386084+JJJYmmm@users.noreply.github.com>
Co-authored-by: 松灵 <26085463+wulipc@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-17 05:01:04 +00:00
Tahsin Tunangemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Luka Govedič
 		cef32104b4 [FP8] Extend per-token-group quantization support to QuantFP8 (#24342) 
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-09-16 18:31:06 -07:00
Matthew BonanniandRobert Shaw d119fc8614 [CI][Bugfix] Fix failing Blackwell test (#24993) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 15:55:02 -07:00
Michael GoinandWentao Ye dbebb7f812 [Perf] Reuse workspace for FP8+FP4 Marlin MoE (#20500) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-09-16 15:45:10 -06:00
Aleksandr MalyshevAleksandr MalyshevGregory ShtrasbergGregory Shtrasberg
 		3053a22b33 fp8 kv cache support fix for torch.compile (#22758) 
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>
 2025-09-16 21:27:11 +00:00
Sage MooreLucas WilkinsonLucas Wilkinsonyewentao256Lucas WilkinsonRobert Shaw
 		567939953b [Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
Chih-Chieh Yang 73cfb3c5ee [Model] Clean up and simplify Mamba2 Metadata Usage in both V0 and V1 (#24331) 
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
 2025-09-16 14:53:43 +00:00
Chen Brucelemon412Harry Mellor
 		7ea5c73ad7 [Feat][EPLB] A novel static EPLB placement strategy for MoE models. (#23745) 
Signed-off-by: bruceszchen <bruceszchen@tencent.com>
Signed-off-by: Chen Bruce <bruceszchen@tencent.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Chen Bruce <cszwwdz@vip.qq.com>
Co-authored-by: lemon412 <lemon412@foxmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-16 10:55:16 +00:00
tomeras91andgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 27fcfe7bcf [Mamba] Support TP>1 with quantization for mamba2 mixer in case n_groups % tp_size == 0 (#24593) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-16 10:51:01 +00:00
Jee Jee Li 04ad0dc275 [benchmark] Add triton version in the moe tuned config (#24769) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-16 14:10:54 +08:00
Saman A. PourandJee Jee Li 238c4c1705 [QWEN NEXT] Fused MoE kernels Optimization configs (#24924) 
Signed-off-by: Saman Keon <samanamp@outlook.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-09-16 13:06:03 +08:00