aoshen02
4eef91c03c
[Model] Support R3 capture with DeepGEMM MegaMoE ( #51831 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-08-12 01:07:07 -07:00
Matej Sirovatka and Jee Jee Li
3ac9525507
[Bugfix][Quantization] Reuse online NVFP4 MoE kernel across reloads ( #50074 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-12 14:06:53 +08:00
Matvei Pashkovskii and Claude
20727e2841
[Hardware][AMD] Enable fused bf16→fp32 router GEMM on ROCm ( #50268 )
...
Signed-off-by: Matvei Pashkovskii <mpashkov@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-12 02:46:13 +00:00
Simon Danielsson
47ececb58e
[Perf][ROCm] Dual-stream decode with hipgraphs ( #48223 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-08-12 09:32:24 +08:00
PikaPikachu
466855a2bf
[ROCm] Enable DeepSeek-V4 on gfx11 ( #47017 )
...
This PR enables DeepSeek-V4 checkpoints on ROCm gfx11/RDNA devices.
Signed-off-by: Letian Kang <Letian.Kang@amd.com >
2026-08-12 08:55:10 +08:00
Rohan Potdar and Claude Opus 4.8
0f0cb918b7
[ROCm][MoE] Fix expert_map vs AITER expert_mask for non-AITER experts under EP ( #49758 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-11 15:33:02 -05:00
Andreas Karatzas and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
36f4630d89
[Bugfix][MoE] Fix fused block-scale orientation ( #50727 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 16:05:38 -04:00
Wentao Ye
fd04edef3e
[Refactor] Delete dead code in models ( #51838 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-11 15:44:26 -04:00
Andreas Karatzas and OpenAI Codex
1ab2801dde
[ROCm] Remove stale SDPA and skinny GEMM workarounds ( #50907 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 14:19:16 -04:00
Julien Debache
52be12cfac
feat: allow shared expert overlapping for FlashInfer one-sided all-to-all ( #50569 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-08-11 10:06:22 -07:00
Jiangyun Zhu and OpenAI Codex
5af7c8dad7
[Bugfix] Align Qwen GDN gates with speculative tokens ( #51812 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 23:35:30 +08:00
pmanczak and Kunshang Ji
5b5eae2394
[Misc] Enable test_silu_mul_fp8_quant_deep_gemm on XPU ( #49444 )
...
Signed-off-by: pmanczak <pawel.manczak@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-11 23:19:14 +08:00
0fb9897df1
[Bugfix][MoE] Support GELU tanh in FlashInfer B12x MoE ( #51819 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 15:07:58 +00:00
Jee Jee Li
dd7cc85f17
Add MoE output contract for MoE tail fusion ( #51407 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-11 22:23:59 +08:00
Fangzhou Ai
1044d19998
[ROCm][DSV4] Preserve native MXFP4 TP8 shard allocation ( #51473 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
2026-08-10 23:57:27 -07:00
Priyansh Jain and Andreas Karatzas
2acb055ed7
[CPU][Zen] Route BF16 MoE inference through zentorch on AMD ( #44201 )
...
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-10 23:20:05 -07:00
Wei Zhao
608c12473f
[Attention] Fix MLA prefill workspace allocation size ( #51733 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-08-11 05:34:30 +00:00
c76a425278
[Kernel] Optimize long-context MLA cache gathers ( #51739 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-10 22:23:43 -07:00
aoshen02 and Roger Wang
99e62b802c
[Bugfix][Model Loader] Defer post-load attention weight processing ( #49519 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-10 19:16:57 -07:00
63ac04a61e
[Kimi-K3] DCP support ( #50484 )
...
Signed-off-by: girasoley <girasoley@inferact.ai >
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
Signed-off-by: 云挚 <ningyunxiao.nyx@antgroup.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com >
Co-authored-by: foraxe <73625538+foraxe@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
2026-08-10 12:54:26 -07:00
8fea1d306f
Fix uniform_random routing simulation to sample without replacement ( #43680 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Signed-off-by: Elvir Crnčević <elvircrn@gmail.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-08-10 15:28:57 +00:00
ba1cdcfcf0
[Model][Quantization] Add Ling-3.0-flash-fp8 support (#51265 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-09 23:09:08 -07:00
Isotr0py
3b4c86e489
[Bugfix][Quantization] Fix fp32 weight scale for mxfp4 quantization and per-expert checkpoint mapping ( #51419 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-09 23:08:52 -07:00
Li, Jiang
7ce84b99ce
[CPU] Restore linear dispatch for small unquantized GEMMs ( #51379 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-08-10 11:29:59 +08:00
Lena Onyshchenko and Kunshang Ji
751f2ccdd3
[Kernel][XPU] Tensor-descriptor operand loads for Triton W8A8 scaled_mm ( #47205 )
...
Signed-off-by: oonyshch <xonyshch@gmail.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-10 00:56:11 +00:00
Isotr0py
073c510c91
[Migration] Migrate bitsandbytes support to OOT plugin ( #43529 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-09 10:27:31 +08:00
djramic
5eaa70dac5
[Bugfix][Quantization] Fix INT8 W8A8 MoE crash in TritonExperts ( #51411 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-08-07 22:06:58 -05:00
Nick Hill
9823714346
[Bugfix] Drop stale layer kwarg from online MXFP4 kernel creation (fix precommit) ( #51442 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-07 13:49:00 -07:00
Wei-Ming Chen
1c94e8dc7d
Add NVFP4 KV 4-over-6 scale search ( #45187 )
...
Signed-off-by: weimingc <17592131+meenchen@users.noreply.github.com >
2026-08-07 13:22:27 -07:00
Walter Beller-Morales and Harry Mellor
70456e5e6f
[Bugfix] when loading weights skip empty expert bias if model does not support them ( #50937 )
...
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-07 12:45:52 -07:00
fxmarty-amd
3518110f2f
[Online quantization] Add online MXFP4 quantization support ( #49347 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-07 12:39:46 -07:00
wangxiyuan
4eccf906ca
[Attention] Mamba attention module refactor - Final part ( #44857 )
2026-08-07 11:27:10 -07:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
vanshbhatia-amd
e229fdbc87
[ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI325X ( #50007 )
...
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com >
2026-08-07 12:58:36 -04:00
Xiaochang Wu and Kunshang Ji
d5aae2b464
Fix ROCm architecture import on non-ROCm platforms ( #51357 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-07 05:32:56 -07:00
Julien Debache
ae934ba8a5
feat: extended EPLB support for Mistral Large 3 and additional MoE backends ( #48355 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-08-07 05:32:52 -07:00
Hank_ and Cyrus Leung
b8db7f4abd
[Bugfix][Quantization] Fix dynamic INT8 W8A8 MoE config being built as W8A16 ( #50833 )
...
Signed-off-by: Hank <hcc.mayday@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-07 01:21:13 -07:00
da788334bc
Support DeepSeek-V4 AMD Quark NVFP4 with emulation kernel ( #47972 )
...
Signed-off-by: Jimmy Adams <Adam_ji@outlook.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: huji <huidong.ji@amd.com >
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: huji <huidong.ji@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-07 01:19:31 -07:00
0de0362ea1
[ROCm][CI] Loosen block-FP8 fused MoE test tolerance for large-K shapes ( #48847 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-07 01:19:26 -07:00
e08111211b
[Kernel] Support Nvfp4 Cutedsl Moe Swiglu-oai and Relu2(non-gated) Activation ( #47106 )
...
Signed-off-by: Tiekai Bi <tiekaib@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-06 17:42:45 -07:00
8170c23c4f
[Quantization] Share online weight scales across TP ( #49764 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Roger Wang <hey@rogerw.io >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-06 16:34:33 -07:00
Taneem Ibrahim
9464529612
[ModelRunner v2] Enable decoder token-wise pooling ( #50931 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-06 22:21:52 +00:00
Wentao Ye
adc3e03517
[Mypy Fix] Mypy fix for "vllm/model_executor/models/[aA][bB]" ( #48977 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-06 19:30:15 +00:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
b38e111d3e
[Attention][MLA] Per-request scheduling for MLA chunked context ( #50613 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 16:45:44 +00:00
Harry Mellor
e7b8d59460
Fully generalise input embedding handling in Transformers modelling backend ( #51247 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-06 16:21:24 +00:00
Ning Xie
62a86318de
[VocabParallelEmbedding] fix extra_repr fields concat ( #51224 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2026-08-06 05:22:18 -07:00
Liangqiusong
1e05b21d61
Remove the XPU branch of topk_softplus_sqrt ( #51242 )
...
Signed-off-by: xiaolong <xiaolong.guo@intel.com >
2026-08-06 12:18:45 +00:00
Lyu Han
22013f74ff
Interns2mobius support ( #51149 )
...
Signed-off-by: Lyu Han <lvhan_028@163.com >
2026-08-06 10:39:35 +00:00
anhtra3889 and Khang Ngo
872fd5973e
[Bugfix][LoRA] Guard TrtLlm BF16 MoE LoRA gate on activation type ( #51002 )
...
Signed-off-by: Anh Tran <anhtra@nvidia.com >
Co-authored-by: Khang Ngo <khngo@nvidia.com >
2026-08-06 18:12:54 +08:00
aoshen02