e77daef89e
[Model] Support DeepSeek-V4.1-Flash ( #56214 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-11 02:11:19 -07:00
siyu and Isotr0py
73fb19151f
[Fast Start] Support fp4 ( #55465 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 13:22:17 +00:00
fe26c705f7
[Model] Add Bailing V3 VL support ( #55921 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Qiming Zhang
49632fb3ac
[CI][XPU] Reject CUDA-IPC weight cache on non-CUDA/ROCm platforms ( #56010 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-09-10 14:05:07 +08:00
Jing Wang
95cf420ac1
[CI][Test][Spec Decode] Fix CI failure of Qwen3 Omni DSpark loader mock ( #56011 )
...
Signed-off-by: Jing Wang <jingwang96@qq.com >
2026-09-08 23:06:15 -07:00
Roderick Wu and mgoin
2c9d68f80b
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ( #54809 )
...
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com >
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 12:45:12 -07:00
Roberto L. Castro and Codex
6ddbab03de
[4/N][warmup][DSv4] Migrate common attention kernels ( #50176 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-08 10:25:48 -04:00
weihan
9ea8f3ffc3
[Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls ( #55307 )
...
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com >
2026-09-08 04:13:09 +00:00
7fa2c63796
[Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture ( #55377 )
...
Signed-off-by: ming <ming@inferact.ai >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-08 04:02:23 +00:00
1713b9866a
[2/N][warmup][DSv4] Migrate sequence and DCP kernels ( #53564 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 09:35:54 -04:00
vhagor and Isotr0py
3dc7a68ce4
[Perf] Extend Qwen Triton warmup to avoid first-request latency spikes ( #54797 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-07 11:46:00 +08:00
yulun and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1f778486fc
[Bugfix][Offloader] Preserve prefetch static-buffer slot ownership ( #54975 )
...
Signed-off-by: Big2Wheel <1404207590@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-07 10:58:44 +08:00
de69e821b7
[ROCm][Perf][DeepSeek V4] Fuse native FP8 shared expert with MXFP4 routed experts ( #53161 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Cursor Agent <agent@cursor.com >
2026-09-06 19:42:21 -07:00
d87a440f88
[Core][MRV2] Support eagle3 spec decode with pipeline parallel ( #50514 )
...
Signed-off-by: Yongqin Wang <yongqinwang@roblox.com >
Co-authored-by: Cursor <noreply@cursor.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-05 11:45:34 +08:00
3f41d102c5
[1/2][Model Runner V2] DBO support, eager mode ( #50945 )
...
Signed-off-by: specture724 <specture724@gmail.com >
Signed-off-by: Anjie Hou <149605198+specture724@users.noreply.github.com >
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-04 14:17:39 -07:00
5690b02c03
[Quantization] Support online quantization with partially pre-quantized checkpoints ( #51392 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-04 11:46:06 -07:00
a69e75b9b6
Fast Start ( #54921 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-04 06:32:06 -07:00
Luyi Xiao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8a728663c1
[Bugfix] Reject tokenizer-less Qwen VL processor init ( #54886 )
...
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-03 21:25:35 -07:00
Nick Hill
e55b93f296
[Core] Deprecate "all" mamba cache mode ( #55041 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-03 08:56:59 -07:00
Luke Alonso
1945a94575
[Bugfix][Tests] Stabilize B12X linear kernel checks ( #54996 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-09-02 18:46:08 +00:00
2fe5cef35e
[Fix] Fix FSE compatibility detection for Quark-produced models ( #54573 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: xuebwang-amd <xuebwang@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-01 12:12:20 -05:00
Andrii Skliar and Andrii Skliar
adebc41b7e
[Mamba] Add FlashInfer ReplaySSM backend ( #52506 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-09-01 10:26:43 -04:00
Subhash Polisetti
4707679cd2
[Bugfix][MiniCPM-V] Route video_embeds to the shared vision parser ( #54633 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
2026-09-01 09:16:07 +00:00
Roberto L. Castro and Codex
e16b5e518d
[1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels ( #50175 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-31 23:26:05 -04:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Connor Carpenter
7a67941c1d
[Rust Frontend][gRPC] Add audio and video media inputs ( #53760 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
2026-08-30 11:31:37 +08:00
Stellar鱼
5182f2705b
[CI/Build] Use file rendezvous for UniProc loader fixtures ( #52367 )
...
Signed-off-by: yu-xin-c <2182712990@qq.com >
2026-08-29 11:53:04 +08:00
Ronald and aoshen02
16d6c376bc
[Bugfix][Models] Register sleep-managed runtime buffers ( #53507 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-08-29 10:32:24 +08:00
Shinichi Hemmi
df14152ac6
[Model] Support speculative decoding method for PLaMo3 ( #54239 )
...
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp >
2026-08-28 22:31:07 +00:00
xuebwang-amd
06569a8696
[ROCm][Quantization][MOE] Enable fused shared experts for block-quantized FP8 ( #53097 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
2026-08-28 09:23:07 -07:00
Matthew Bonanni and OpenAI Codex
de9250ac9e
[Attention] Enable masked MHA for GLM-5 head dimensions ( #53785 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-27 21:17:12 +00:00
Nick Hill
4aab2b0ebe
[Model Runner V2] Use MRV2 for all models by default ( #53183 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-27 11:59:34 -07:00
Luke Alonso
3454335f7b
[Bugfix] Resolve B12X modules before Dynamo tracing ( #53326 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-08-25 09:20:15 -07:00
Taneem Ibrahim
79bb395eea
[Pooling] Improve BGE-M3 sync pooling throughput by up to 3.13% ( #53464 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-24 08:33:32 -04:00
Wyett
b26039b09f
Add routed expert loading for gpt-oss ( #52209 )
...
Signed-off-by: Wyett <wyettzeng@gmail.com >
Signed-off-by: wyett <wyettzeng@gmail.com >
2026-08-23 17:59:08 +08:00
Zhou248
2f55ef254c
[Model] Add Qwen3-Omni DSpark support ( #52560 )
...
Signed-off-by: Zhou248 <630563665@qq.com >
2026-08-22 15:35:22 -07:00
Shuolei Wang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
236f78cc5c
[Core][RL] Support sparse checkpoint updates through native weight loaders ( #50723 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-22 17:39:58 +00:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟
b2db227a7c
[Bugfix][R3] Unwrap UniformTypeKVCacheSpecs when selecting the routed-experts KV group ( #53240 )
...
Signed-off-by: Hollow Man <hollowman@opensuse.org >
2026-08-21 19:13:53 -07:00
Luke Alonso and mgoin
2740c817ff
[Kernel] Add b12x FP4 MoE backend ( #52018 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-08-21 08:04:36 -07:00
Harry Mellor
f0c14b4f77
Fix weight tying ( #51665 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-20 10:31:55 +00:00
Qiming Zhang
754e1c3de4
[CI][XPU] Skip test_fused_shared_expert.py on XPU ( #53035 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-08-20 14:56:09 +08:00
b05ae5dc00
[CI][Bugfix] Complete DeepSeek-V4 FSE test fixture contract ( #52842 )
...
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com >
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-18 19:04:10 -06:00
Summer Yang
5f7a20b316
[nv] add pcp support in dsv3.2 ( #52046 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-18 14:59:08 -07:00
Woosuk Kwon and OpenAI Codex
01e56caaf2
[Bugfix][MLA] Do not use Dense MHA for GLM-5.2 ( #52512 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 11:20:48 -07:00
fxmarty-amd
88b2bff2c6
[MOE] Standardize and abstract fused shared expert optimization selection ( #51695 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-18 12:56:56 -04:00
Roberto L. Castro and Codex
49905ad94d
[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup ( #50174 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-17 10:11:14 -07:00
Misha Goin
4ab5e5012a
[Refactor] Simplify B12X linear kernels and warmup ( #52368 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-17 09:45:45 -07:00
Qiming Zhang and Andreas Karatzas
1f7427bc0a
[UT][XPU] fix b12x UT ( #52265 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-14 09:49:47 -04:00
Luke Alonso
3c79b1a8bf
[Kernel] Add B12X dense linear backends ( #52016 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-08-13 19:48:33 -07:00
aoshen02 and Codex
f2906b492e
[Bugfix] Avoid repeated layerwise reload warning scans ( #49505 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-08-12 13:53:50 +08:00
