f727951d3f
[Bugfix] Re-land MiniMax M3 default video processor ( #50305 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-07-31 07:46:18 +00:00
10e6b40015
[CPU][BugFix] Remove redundant kv cache write ( #50437 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-31 15:28:44 +08:00
Mi Jiazhi and Jiazhi Mi
3ee2bd1321
Fix duplicate HunyuanVL image boundary tokens ( #49691 )
...
Co-authored-by: Jiazhi Mi <jiazhimi@tencent.com >
2026-07-31 07:20:06 +00:00
Isotr0py
1180b604f8
[Multimodal] Expose mm hash algothrim selection to cli args ( #49686 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-31 05:25:19 +00:00
Taneem Ibrahim
0f17394564
[Model Runner V2] Enable encoder token classification ( #50293 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-07-31 04:15:36 +00:00
Sebastian Woo and Claude
bebf918044
[Bugfix][Model] Reject encoder-backbone jina-embeddings-v5 checkpoints with a clear error ( fixes #50337 ) ( #50352 )
...
Signed-off-by: seewoo <seewoo@ucsc.edu >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-31 11:43:31 +08:00
b49eaf205a
[DSv4] Remove sparse-MLA q-head padding for FlashInfer >=0.6.14 ( #48047 )
...
Signed-off-by: Jeff Ma <jeffjma@umich.edu >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-07-31 03:30:21 +00:00
Chang Guo and Cursor Agent
541128bdeb
[KV Offload] Enable single-copy MLA layout for CPUOffloadingSpec ( #50301 )
...
Signed-off-by: Change72 <changg@nvidia.com >
Co-authored-by: Cursor Agent <cursor-agent@cursor.com >
2026-07-31 06:20:47 +03:00
Xiaochang Wu and Kunshang Ji
1d8be5cb44
[XPU] [BugFix] Add deepseek_v4_fp8 to xpu supported_quantization list ( #50434 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-31 11:12:04 +08:00
Ma Jian and Kunshang Ji
ef0d084b4b
[XPU] Fix FP8 block scale layout for MLA compatibility ( #50349 )
...
Signed-off-by: Ma Jian <jian1.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-31 11:05:35 +08:00
6724051c57
[CI/Build][AMD] Install triton_kernels via CMake ( #50328 )
...
Signed-off-by: Ryan Rock <ryan.rock@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: depthfirst-app[bot] <184448029+depthfirst-app[bot]@users.noreply.github.com>
Co-authored-by: root <root@smci355-ccs-aus-m11-05.cs-aus.dcgpu >
2026-07-30 22:05:19 -05:00
vanshbhatia-amd
4689c7dd61
[ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI325X ( #50006 )
...
Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com >
2026-07-31 02:43:53 +00:00
Bugen Zhao
ab98034d4c
[Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness ( #50420 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-31 10:02:17 +08:00
f1899b2ffd
[Bugfix][ROCm] AITER MLA: size MTP verification decode metadata for real qlen/dtype ( #45227 )
...
Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io >
Signed-off-by: Chaemin Lim <chaemin.lim@mangoboost.io >
Signed-off-by: QinPR <1905873179@qq.com >
Co-authored-by: Edwin Lim <edwin.lim@mangoboost.io >
Co-authored-by: limeward <32970461+edwinlim0919@users.noreply.github.com >
Co-authored-by: QinPR <1905873179@qq.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-07-30 20:50:34 -05:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
Wentao Ye
0bff0ce5d3
[Kimi K3 Bug] Fix deepgemm support for kimi k3 ( #50458 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 20:10:16 -04:00
Giancarlo Delfin
dec13a33b7
[Model Runner V2][Spec Decode] Add multi-layer MTP speculator ( #48892 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-07-30 15:54:15 -07:00
Ryan Clark and Claude Fable 5
68fb303a41
[Bugfix] Preserve Marlin runtime tensor storage across weight reload ( #48438 )
...
Signed-off-by: Ryan Clark <ryanclark2k@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-30 14:50:03 -07:00
Shangdi Yu and OpenAI Codex
45b60e3914
[Kernel][Helion] Disable unsafe B200 RMS reduction warp specialization ( #50345 )
...
Signed-off-by: Shangdi Yu <shangdiy@meta.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 20:37:52 +00:00
Netanel Haber and OpenAI Codex
70bd10930b
[Quantization] Honor --linear-backend for ModelOpt W4A16 ( #50273 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 13:19:56 -07:00
Richard Zou and Codex
c27b080d26
[compile] Fix fake kernel return dtype ( #50444 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-07-30 19:50:02 +00:00
Andreas Karatzas
3f90c7e9e6
[ROCm] Pass pointers to FlyDSL MoE kernels ( #50378 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 12:49:26 -07:00
Andreas Karatzas and OpenAI Codex
30e333ca5f
[Bugfix] Shut down private Tensorizer engines ( #49840 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 12:33:22 -07:00
Bugen Zhao
629a938a92
[Frontend] Preserve bare Inkling text in Python and Rust parsers ( #50403 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-30 12:25:10 -07:00
12a34a6bc7
[ROCm][DSV4] B-preshuffle the attention fp8 projections ( #46720 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 13:59:52 -05:00
Netanel Haber and OpenAI Codex
61cacd272f
[Bugfix][MoE] Write Humming results to the supplied output buffer ( #50338 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 11:34:10 -07:00
Wentao Ye
837eae6458
[DSv4 Perf] Remove redundant full kernel for dsv4, 1.88x kernel performance improvement ( #50298 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 11:39:07 -04:00
Wentao Ye
904fae8be1
[DSv4 Perf] Fix redundant memory allocation and copy for dsv4 pp buffer, 448 MiB GPU memory saved ( #50312 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-30 11:38:51 -04:00
Andreas Karatzas
5b95890742
[FlexAttention] Avoid encoder block-mask compile explosion ( #50339 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 11:24:13 -04:00
Aaron Hao
30b4e7f479
[rl] Stateful Trainer Send: IPC [2/N] ( #48981 )
...
Signed-off-by: haoaaron <ahao@anyscale.com >
2026-07-30 21:59:08 +08:00
BadrBasowid
59e831c09a
[Compilation]Fuse Transformers Residual Add + RMSNorm ( #48757 )
...
Signed-off-by: BadrBasowid <badr.basowid@gmail.com >
2026-07-30 13:46:36 +00:00
Julien Denize
1a20d23dab
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls ( #48947 )
2026-07-30 15:38:00 +02:00
e2efe79695
[ROCm]Migrating Deepseek V3.2 to vllm/models/deepseek_v32/ ( #47207 )
...
Signed-off-by: sroberts-amd <sroberts@amd.com >
Signed-off-by: Stacy Roberts <sroberts@amd.com >
Co-authored-by: Elvir Crncevic <elvircrn@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 08:37:43 -05:00
Li, Jiang and Claude
072a472782
[CPU] Bump up CPU kernels to latest version ( #50387 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-30 19:18:43 +08:00
+13
aeeb36b1f1
[New model] Kimi K3 ( #50000 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Summer Yang <girasoleyang@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: Bowen Wang <abmfy@icloud.com >
Co-authored-by: gnovack <novackgm@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-07-30 03:49:58 -07:00
Andreas Karatzas
e04a30a77c
[Frontend] Lazily initialize chat media connectors ( #49914 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-07-30 14:56:37 +08:00
yinfengLiu
f1e8fd27ae
[ROCm] Add AITER FP8 ViT encoder attention ( #49937 )
...
Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com >
2026-07-30 06:22:10 +00:00
stefankoncarevic and Andreas Karatzas
b88916617d
[ROCm] [CI] Support cached K/V (key/value=None) in Triton prefix-prefill ( #48257 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-29 23:31:51 -05:00
lkk and Kunshang Ji
e5f48dfda2
[Quantization][Autoround][XPU] Add W4A16(moe) / MXFP4(linear/moe) Support ( #47124 )
...
Signed-off-by: lkk12014402 <kaokao.lv@intel.com >
Signed-off-by: lkk <33276950+lkk12014402@users.noreply.github.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-07-30 09:52:13 +08:00
4e582c5b54
[PD][Bugfix] Rebase KV lease deadlines onto worker clock ( #50326 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: esmeetu <jasonailu87@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-07-30 01:25:42 +00:00
Nick Hill and Claude Opus 5
437e0b7f8e
[BugFix] Fix P/D preemption race condition ( #50297 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-07-30 01:11:18 +00:00
Martin Hickey
0a31372e5f
[Frontend] Add detokenization streaming derender for disaggregated serving ( #47301 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-07-29 18:01:20 -07:00
bugkeep and Claude
48aa8d8d75
[Bugfix] Prevent stale multiproc RPC deadlines from becoming unbounded waits ( #41357 )
...
Signed-off-by: bugkeep <1921817430@qq.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-29 16:41:34 -07:00
christian
9d1aa4dda8
[Frontend] Add diarized_json support for MOSS-Transcribe-Diarize ( #48543 )
...
Signed-off-by: Lucas Christian <lucaschgf7@gmail.com >
2026-07-29 23:24:30 +00:00
5fa0154448
[Rubin] Enable NVLink all-reduce paths on SM107 ( #49647 )
...
Signed-off-by: Zachary Aristei <zaristei@nvidia.com >
Co-authored-by: Sahithi Chigurupati <chigurupati.sahithi@gmail.com >
Co-authored-by: hlu1 <14827759+hlu1@users.noreply.github.com >
2026-07-29 20:53:25 +00:00
Connor Carpenter and Nick Hill
48fc2e2797
feat(grpc): add KV event source discovery ( #50033 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-29 13:46:40 -07:00
Nick Hill
2ecd8645d8
Revert "[Misc][Minimax-M3]add default video_processor ( #50092 )" ( #50313 )
2026-07-29 13:46:07 -07:00
Wentao Ye
82642d7d6c
[Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance improvement ( #49750 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-07-29 13:45:38 -07:00
roikoren755
93477454f9
[torch.compile] Compile CustomOp.forward_native for ReLU^2 to avoid raw torch ops inside opaque custom ops ( #50244 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
2026-07-29 18:27:31 +00:00
Chanh Nguyen and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f98061ce6c
fix(step3p5-mtp): honor exclude_modules for the MTP head via prefix ( #48883 )
...
Signed-off-by: Chan Nguyen <channguyen@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-29 16:58:29 +00:00
