Dr Tasos Varoudis and Lucas Wilkinson
c416f15710
[Model] Fix Kimi-K3 MLA with disabled context parallelism ( #50404 )
...
Signed-off-by: Tasos Varoudis <varoudis@archtech.gr >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-05 02:33:14 +00:00
Andreas Karatzas
33c50587d2
[ROCm] Restore Inkling MTP backend parity ( #50806 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-08-05 02:02:57 +00:00
xiaozhoupy and Claude
3756bf1e2b
[Kernel][SM100] Add a CuTeDSL fused query kernel ( #49792 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-04 15:23:06 -07:00
Canlin Guo
edbc4969a7
[Kernel][Inkling] Fuse shared-expert partial addition into the Lamport collective ( #50697 )
...
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
2026-08-04 18:41:35 +00:00
Wentao Ye
d31de3c421
[Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved ( #50912 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-04 14:00:44 -04:00
7ac2ec7582
[Kimi-K3][AMD] Fuse AttnRes state updates and norms ( #50593 )
...
Signed-off-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com >
Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 12:32:13 -04:00
namgyu-youn and Claude Opus 5
41ba11b841
[Bugfix][Kimi-K3] Enforce packed rows and op availability in AttnRes dispatch ( #50567 )
...
Signed-off-by: namgyu-youn <namgyu.dev@gmail.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-08-03 17:13:33 -07:00
c810937573
[Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 ( #49791 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-03 16:49:41 -07:00
Raphaël Rialland
4a3447d200
[Bugfix][Model Runner V2] Restore multimodal draft capability detection ( #50417 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
2026-08-03 14:18:14 -07:00
5df9999fcf
[Kimi-K3] Add option to shard the shared expert instead of replicating ( #50656 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-03 11:41:24 -04:00
wangxiyuan
dd11df04f3
[Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation ( #49389 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-08-03 11:15:13 -04:00
Wentao Ye
0cf49a5d15
[Refactor] Remove multiple dead codes ( #50285 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-03 10:59:08 -04:00
Jee Jee Li
b9d1e2437e
K3: Move LatentMoERunner ( #50678 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-03 09:36:17 +00:00
Jee Jee Li
4635cc3e8f
Shard the K3 Latent-MoE up-projection on large batches ( #50383 )
2026-08-03 00:49:39 -07:00
Jason Li
0055b8bfa3
[Attention][MiniMax-M3] Add MSA speculative decode verification ( #50032 )
2026-08-02 13:46:55 -07:00
Fangzhou Ai and OpenAI Codex
c666810676
[ROCm][Bugfix][Kimi-K3] Preserve MoE correction bias in FP32 ( #50761 )
...
Signed-off-by: Fangzhou Ai <Fangzhou.Ai@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-02 20:30:54 +00:00
Woosuk Kwon
38a466e7b6
[DSV4] Implement Sequence Parallelism ( #46789 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-08-01 10:22:33 -07:00
Andreas Karatzas and OpenAI Codex
f7097a92fd
[Bugfix][CI] Prevent common ops imports from initializing CUDA ( #50639 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-01 00:13:29 -04:00
Kyle Sayers
d87d2ca747
[Compressed-Tensors] Support Kimi-K3 quantized models ( #50500 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-07-31 17:06:30 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
df71917cf1
[DSv4 Perf] Optimize workspace reuse for eager break, 3.9% E2E TTFT improvement. ( #49236 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-31 08:45:58 -07:00
Jee Jee Li
92643d68f5
K3 DSpark AR fusion ( #50242 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-07-31 11:12:03 +00:00
bnellnm and Harry Mellor
6e311c6e20
[MoE Refactor] Rename FusedMoE to FusedMoEFactory ( #44941 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-31 11:28:18 +01:00
Thien Tran and Codex
34bb795ff3
[CI] Add M3 MSA tests to CI ( #49143 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <noreply@openai.com >
2026-07-31 01:01:48 -07:00
f727951d3f
[Bugfix] Re-land MiniMax M3 default video processor ( #50305 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-07-31 07:46:18 +00:00
b49eaf205a
[DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 ( #48047 )
...
Signed-off-by: Jeff Ma <jeffjma@umich.edu >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-07-31 03:30:21 +00:00
Giancarlo Delfin
dec13a33b7
[Model Runner V2][Spec Decode] Add multi-layer MTP speculator ( #48892 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-07-30 15:54:15 -07:00
12a34a6bc7
[ROCm][DSV4] B-preshuffle the attention fp8 projections ( #46720 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 13:59:52 -05:00
Wentao Ye
837eae6458
[DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement ( #50298 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 11:39:07 -04:00
Wentao Ye
904fae8be1
[DSv4 Perf] Fix redundant memory allocation and copy for dsv4 pp buffer, 448 MiB GPU memory saved ( #50312 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 11:38:51 -04:00
e2efe79695
[ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ ( #47207 )
...
Signed-off-by: sroberts-amd <sroberts@amd.com >
Signed-off-by: Stacy Roberts <sroberts@amd.com >
Co-authored-by: Elvir Crncevic <elvircrn@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 08:37:43 -05:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
Nick Hill
2ecd8645d8
Revert "[Misc][Minimax-M3]add default video_processor ( #50092 )" ( #50313 )
2026-07-29 13:46:07 -07:00
stefankoncarevic
381b691620
[ROCm][CI] Fix Kimi K3 KDA on ROCm ( #50262 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
2026-07-29 15:47:50 +00:00
rongfu.leng and Cyrus Leung
df2735ea2e
[Misc][Minimax-M3]add default video_processor ( #50092 )
...
Signed-off-by: rongfu.leng <lenronfu@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-29 01:24:51 -07:00
+12
7c6729b769
[Model] Add Kimi K3 support: model files and kernels [1/N] ( #50089 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
2026-07-29 14:10:58 +08:00
17a74b745b
[Model] Add Inkling compressed-tensors dynamic FP8 support ( #48876 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-28 19:27:01 -07:00
Yiliu Dong and OpenAI Codex
ba702e978e
[Attention] Skip sparse indexer scoring for dense short prefills ( #48407 )
...
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-28 16:17:22 +00:00
Harry Mellor
62d8db7c05
[Bugfix] Add missing vllm/models/kimi_k3/__init__.py ( #50131 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-28 13:47:04 +00:00
Thien Tran
61ac368021
[Kimi-K3] Add AttnRes kernels ( #50090 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-07-28 00:22:09 -07:00
Colin Z and Andreas Karatzas
fbb1ef6803
[Bugfix] Fix DeepseekV4FP8 Quark MXFP4 crash on list-valued weight ( #49634 )
...
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-27 22:32:42 -05:00
Wentao Ye
b2f9e4caa4
[DSv4 Perf] Adaptive topk width, 1.0% E2E throughput improvement ( #50004 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-27 15:56:52 -04:00
TJian and OpenAI Codex
0906123953
[ROCm] [Model] Enable TML inkling ( #48841 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-27 10:05:46 +00:00
Wentao Ye
7fe6d3c76b
[Perf] Fix moe reduce_scatter perf regression by removing additional comm, 5% E2E throughput gain back. ( #48763 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-25 16:36:19 +00:00
rongfu.leng
d1a8ba63d9
[Bugfix][MiniMax-M3] Fix token-major top-k buffer handling in Triton … ( #49149 )
...
Signed-off-by: rongfu.leng <lenronfu@gmail.com >
2026-07-25 06:45:27 -07:00
Woosuk Kwon
213f681f81
Revert "[Perf][GLM-5.2] Blackwell decode optimizations" ( #49768 )
2026-07-24 16:04:59 -07:00
dd72658e7d
[Perf][GLM-5.2] Blackwell decode optimizations ( #48597 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Signed-off-by: zhou <zhou@zhoudeMacBook-Pro.local >
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zhou <zhou@zhoudeMacBook-Pro.local >
Co-authored-by: stefankoncarevic <Stefan.Koncarevic@amd.com >
2026-07-23 21:36:27 -07:00
Kyle Sayers
bf27e34ebb
[CompressedTensors] DeepSeek4 CT Quantization Support ( #41276 )
...
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com >
2026-07-23 20:07:31 -07:00
Michael Goin
d65acd83d8
[Model] Support llm-compressor Inkling NVFP4 weights ( #49258 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-07-23 20:06:01 -07:00
Wentao Ye
b0cb1da1bd
[DSv4 Perf] Skip topk and router when not needed, 3.4% E2E TTFT improvement for Decode case ( #49486 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-23 13:08:08 -04:00
Mike G
76bf55240c
[Bugfix] Fix DeepSeek-V4 DSpark draft shared-expert padding for TP > 8 ( #49415 )
...
Signed-off-by: Mike G <180722391+mikekg@users.noreply.github.com >
2026-07-23 05:06:21 +00:00