Commit Graph
2438 Commits
Author SHA1 Message Date
Nico Holmberg 7b54690244 [ROCm][Perf] Expose AITER MoE sorting dispatch policy via env var (#39177)
Signed-off-by: nholmber <nholmber@users.noreply.github.com>
2026-05-27 13:11:02 +08:00
Andreas Karatzas 5bdb181df5 [ROCm][CI] Fix ROCm multimodal Qwen2.5-VL activation compile and Phi4MM ragged image mask handling (#43647)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-05-26 19:53:34 -07:00
Jee Jee Li 6e503868ca [Kernel] Porting fuse_minimax_qk_norm to manual fusion (#43410)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-05-26 13:16:03 -07:00
Wei-Ming Chen 6f5b533241 Add LM head quantization support for ModelOpt (#42124)
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
2026-05-26 09:21:05 -07:00
bnellnmandRobert Shaw f51bbc694d [MoE Refactor] W4a8 int8 oracle (#42789)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-05-26 11:15:42 -04:00
bnellnmandRobert Shaw b226ddacfd [MoE Refactor] Migrate ModelOptMxFp8FusedMoE to oracle (#42768)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-05-26 11:14:14 -04:00
Mohammad Miadh Angkad a970fb5a1a Fix CuPy runtime deps and restore humming (#43530)
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-05-26 05:59:40 -07:00
Hank_ b3269454b1 [chores][log] change registry log from warning to debug (#43045)
Signed-off-by: Hank <hcc.mayday@gmail.com>
2026-05-26 00:13:46 -07:00
Thien Tran d56612c621 [GDN] GDN Prefill kernel for SM100 (#43273)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
2026-05-26 14:02:11 +08:00
Jee Jee Li ec5de7fa7d [LoRA] Add one shot triton kernel For MoE LoRA (#42290)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
2026-05-25 19:47:04 -07:00
Jee Jee Li d4004455d2 [Kernel] Remove NormGateLinear (#43554)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-05-25 09:49:19 +00:00
weizhoublue 6cbe448eed fix: MoE model using shared routed experts crashes on AMD GPUs (#42373)
Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>
2026-05-25 12:03:05 +08:00
Jee Jee Li b06813e872 [Kernel] Add mhc_pre_big_fuse_with_norm_tilelang (#43474)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
2026-05-25 01:19:45 +00:00
daniserebandBanani Ghosh d56285c747 Tuning script and configs for Triton Mamba SSU kernel (#43083)
Signed-off-by: Banani Ghosh <bg2502@nyu.edu>
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
Co-authored-by: Banani Ghosh <bg2502@nyu.edu>
2026-05-24 20:12:44 +03:00
TJian 1806d1adfc [ROCm] [DSv4] [Perf] Support DeepSeek v4 MTP (#43385)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
2026-05-24 18:43:08 +08:00
Michael Goin 10d264a2b9 Revert "[Misc] add humming to dependencies" (#43492) 2026-05-23 14:21:13 -07:00
Siddharth BedekarandOpenAI Codex 4438b6e7dc [MoE] Migrate W4A8 CT to oracle kernel setup (#42680)
Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-05-23 13:56:01 -04:00
Wei-Ming Chen 09a219c075 [ModelOpt] Support Qwen3.5/3.6 VLM quantized prefix mapping (#42546)
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com>
2026-05-23 06:23:31 +00:00
Taneem Ibrahim 3a1c062151 [Misc] Added missing return type annotations to improve mypy and IDE tooling (#43383)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-05-23 13:28:22 +08:00
Qiming ZhangandKunshang Ji 54d153637b [XPU] reudce host overhead of XPU MOE (#42915)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-23 13:09:34 +08:00
Ma JianandKunshang Ji a5bbd81e2e [XPU]feat: enable FP8 block-scaled quantization on XPU (#42952)
Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-23 12:33:18 +08:00
Andreas Karatzas d28bdf9344 [ROCm][CI] Fix ROCm LoRA Transformers fallback with full CUDA graphs (#41577)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-05-23 04:31:32 +00:00
Itay Alroy 6d30655b13 elastic_ep: stage/commit MoE quant method on reconfigure (#40881)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
2026-05-22 18:57:26 -04:00
Ma JianandKunshang Ji 8de5cabeb7 [XPU]fix: add XPU platform guards to DeepSeek-V4 ops (#42950)
Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-23 06:29:45 +08:00
Juhi Mittal e203006a8b [Quantization][ModelOpt] W4A16 NVFP4 fused MoE + mixed-precision dispatch (#42566)
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
2026-05-22 20:51:49 +00:00
haosdentandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 4e597b7491 [Bugfix] Clear error message for FP8 torchao quantization on unsupported GPUs (#36854)
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-22 20:09:17 +00:00
Artem Perevedentsev 23f7b11bf4 [Bugfix] Detect wrong libcute_dsl_runtime.so variant in FlashInfer GDN (#43427)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-05-22 19:33:33 +00:00
sychen52 fb21d8b4f9 Add NVFP4 MOE support for Deepseek V4. (#42209)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
2026-05-22 07:21:51 -07:00
b3c7ffcab8 [Misc] Replace assert with proper exceptions for security and validation in pooling (#43286)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-05-22 18:43:33 +08:00
Ma JianandKunshang Ji d3d1cf6972 [XPU]feat: add XPU fallback for MoE topk routing and MXFP4 backend (#42951)
Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-22 10:22:45 +00:00
wangxiyuan 7e1b45a092 [Attention] Mamba attention module refactor (#41126)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
2026-05-22 17:13:12 +08:00
Xiaochang WuandKunshang Ji 8c8b1825eb [XPU] Enable multiple key kernels for sparse attention (#37888)
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com>
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-22 12:02:51 +08:00
Chengze FanandLu Fang 2998a047aa [Bugfix] Fix DSV4 Base model swiglu limit issue in FP8 path (#42855)
Signed-off-by: Chengze Fan <chengze@meta.com>
Signed-off-by: Chengze Fan <fancz2002@gmail.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-05-21 19:43:01 -07:00
Wentao Ye b29cbf0652 [Perf] zeros -> empty to remove additional fill (#42988)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-05-21 16:00:29 -04:00
Wentao YeandIsotr0py 68e07d5916 [Bug] Fix ci issue assert output_size is not None AssertionError (#43261)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-05-21 16:58:09 +08:00
zhangxin81 edafea3555 Fix FlashInfer TRTLLM NvFP4 monolithic MoE routing (#43223)
Signed-off-by: zhangxin81 <115389973+zhangxin81@users.noreply.github.com>
2026-05-21 01:17:12 -07:00
zexplorerhjandzexplorerhj b719b1635b Update KDA chunk prefill decay to use exp2 semantics (#43195)
Signed-off-by: zexplorerhj <19794632+zexplorerhj@users.noreply.github.com>
Co-authored-by: zexplorerhj <19794632+zexplorerhj@users.noreply.github.com>
2026-05-21 01:16:27 -07:00
e45df8c3f7 [Bugfix] Fix Qwen3.5 GatedDeltaNet in_proj_ba Marlin failure at TP>=2 (#36329)
Signed-off-by: Adi McM Sonus Flow <biuro@sonusflow.pl>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 21:22:01 -07:00
Michael Goin 5774aad9c5 [Perf][gpt-oss] Downgrade triton_kernels to v3.5.1 (#43135)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-05-20 14:13:12 -07:00
meena-at-workandClaude Sonnet 4.6 363fc84407 Integrate flashinfer b12x MoE and FP4 GEMM kernels for SM120/121 (#40082)
Signed-off-by: Meenakshi Venkataraman <meenakshiv@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 17:21:11 +00:00
Dipika Sikka df84fb07a6 Remove additional dead code as a follow-up to #42889 (#43144)
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
2026-05-20 10:01:45 -04:00
Artem Perevedentsev 1cb224430b [GDN] Enable FI Blackwell GDN prefill kernel (#40717)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-05-20 01:46:55 -07:00
Max de Baysergemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>wang.yuqi
117afeea46 Fix error in Dynamic NTK scaling (#41277)
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-05-19 17:27:54 -04:00
Jinzhen Lin 8200fbe1ac [Misc] add humming to dependencies (#42540)
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
2026-05-19 08:36:47 -07:00
Taneem Ibrahim 4a4fdabe28 [Misc] Aligning tokwise pooler heads for consistency (#43041)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
2026-05-19 06:16:42 +00:00
Woosuk Kwon 87b08c5f64 [Model Refactoring] Move DeepSeek V4 layers to models/deepseek_v4/ [2/N] (#43039)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-05-18 21:00:58 -07:00
Kunshang Ji 36dcaf25d8 [XPU] add gptq(int4) support (#37844)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-19 11:17:09 +08:00
Woosuk Kwon 287471b994 [Model Refactoring] Migrate DeepSeek V4 to vllm/models/ [1/N] (#43004)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
2026-05-18 19:50:02 -07:00
Wentao Ye cd49a05d5a [Refactor] Remove dead code (#42889)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-05-18 16:41:22 -04:00
Tuukka Sarvi 8fc1c284b9 [ROCm] Guard AITER GDN decode fast path by layout (#42880)
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
2026-05-18 11:56:22 -07:00