+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Connor Carpenter
7a67941c1d
[Rust Frontend][gRPC] Add audio and video media inputs ( #53760 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
2026-08-30 11:31:37 +08:00
Stellar鱼
5182f2705b
[CI/Build] Use file rendezvous for UniProc loader fixtures ( #52367 )
...
Signed-off-by: yu-xin-c <2182712990@qq.com >
2026-08-29 11:53:04 +08:00
Ronald and aoshen02
16d6c376bc
[Bugfix][Models] Register sleep-managed runtime buffers ( #53507 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-08-29 10:32:24 +08:00
Shinichi Hemmi
df14152ac6
[Model] Support speculative decoding method for PLaMo3 ( #54239 )
...
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp >
2026-08-28 22:31:07 +00:00
xuebwang-amd
06569a8696
[ROCm][Quantization][MOE] Enable fused shared experts for block-quantized FP8 ( #53097 )
...
Signed-off-by: xuebwang-amd <xuebwang@amd.com >
2026-08-28 09:23:07 -07:00
Matthew Bonanni and OpenAI Codex
de9250ac9e
[Attention] Enable masked MHA for GLM-5 head dimensions ( #53785 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-27 21:17:12 +00:00
Nick Hill
4aab2b0ebe
[Model Runner V2] Use MRV2 for all models by default ( #53183 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-27 11:59:34 -07:00
Luke Alonso
3454335f7b
[Bugfix] Resolve B12X modules before Dynamo tracing ( #53326 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-08-25 09:20:15 -07:00
Taneem Ibrahim
79bb395eea
[Pooling] Improve BGE-M3 sync pooling throughput by up to 3.13% ( #53464 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-24 08:33:32 -04:00
Wyett
b26039b09f
Add routed expert loading for gpt-oss ( #52209 )
...
Signed-off-by: Wyett <wyettzeng@gmail.com >
Signed-off-by: wyett <wyettzeng@gmail.com >
2026-08-23 17:59:08 +08:00
Zhou248
2f55ef254c
[Model] Add Qwen3-Omni DSpark support ( #52560 )
...
Signed-off-by: Zhou248 <630563665@qq.com >
2026-08-22 15:35:22 -07:00
Shuolei Wang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
236f78cc5c
[Core][RL] Support sparse checkpoint updates through native weight loaders ( #50723 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-22 17:39:58 +00:00
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟
b2db227a7c
[Bugfix][R3] Unwrap UniformTypeKVCacheSpecs when selecting the routed-experts KV group ( #53240 )
...
Signed-off-by: Hollow Man <hollowman@opensuse.org >
2026-08-21 19:13:53 -07:00
Luke Alonso and mgoin
2740c817ff
[Kernel] Add b12x FP4 MoE backend ( #52018 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-08-21 08:04:36 -07:00
Harry Mellor
f0c14b4f77
Fix weight tying ( #51665 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-20 10:31:55 +00:00
Qiming Zhang
754e1c3de4
[CI][XPU] Skip test_fused_shared_expert.py on XPU ( #53035 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-08-20 14:56:09 +08:00
b05ae5dc00
[CI][Bugfix] Complete DeepSeek-V4 FSE test fixture contract ( #52842 )
...
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com >
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-18 19:04:10 -06:00
Summer Yang
5f7a20b316
[nv] add pcp support in dsv3.2 ( #52046 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-18 14:59:08 -07:00
Woosuk Kwon and OpenAI Codex
01e56caaf2
[Bugfix][MLA] Do not use Dense MHA for GLM-5.2 ( #52512 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 11:20:48 -07:00
fxmarty-amd
88b2bff2c6
[MOE] Standardize and abstract fused shared expert optimization selection ( #51695 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-18 12:56:56 -04:00
Roberto L. Castro and Codex
49905ad94d
[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup ( #50174 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-17 10:11:14 -07:00
Misha Goin
4ab5e5012a
[Refactor] Simplify B12X linear kernels and warmup ( #52368 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-17 09:45:45 -07:00
Qiming Zhang and Andreas Karatzas
1f7427bc0a
[UT][XPU] fix b12x UT ( #52265 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-14 09:49:47 -04:00
Luke Alonso
3c79b1a8bf
[Kernel] Add B12X dense linear backends ( #52016 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-08-13 19:48:33 -07:00
aoshen02 and Codex
f2906b492e
[Bugfix] Avoid repeated layerwise reload warning scans ( #49505 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-08-12 13:53:50 +08:00
Richard Zou
cb30f6f8dd
[Testing] Fix test_sharded_state_loader ( #51736 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-08-11 21:16:09 +00:00
Andreas Karatzas and OpenAI Codex
1ab2801dde
[ROCm] Remove stale SDPA and skinny GEMM workarounds ( #50907 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 14:19:16 -04:00
6c95a641e9
[2/N][Feat][Perf] Add new warmup infrastructure for JITs. Add predicate filtering for JIT warmup, and migrate Inkling FA4 ( #49315 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 07:46:07 -04:00
aoshen02 and Roger Wang
99e62b802c
[Bugfix][Model Loader] Defer post-load attention weight processing ( #49519 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-10 19:16:57 -07:00
Taneem Ibrahim
9464529612
[ModelRunner v2] Enable decoder token-wise pooling ( #50931 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-06 22:21:52 +00:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
aoshen02 and yihengz
6153dbe036
[R3] Unify routed expert shape configuration ( #50940 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: yihengz <yihengz@nvidia.com >
2026-08-04 20:20:41 -07:00
aoshen02 and aoshen02
7153fd70f6
[Bugfix][MoE] Filter packed expert weights during EP loading ( #49558 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-08-04 16:32:03 +00:00
aoshen02 and Nick Hill
42ab184ea7
[MRV2] Enable routed-experts capture ( #50721 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-03 21:00:13 +00:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
Ryan Clark and Claude Fable 5
68fb303a41
[Bugfix] Preserve Marlin runtime tensor storage across weight reload ( #48438 )
...
Signed-off-by: Ryan Clark <ryanclark2k@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-30 14:50:03 -07:00
Andreas Karatzas and OpenAI Codex
30e333ca5f
[Bugfix] Shut down private Tensorizer engines ( #49840 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 12:33:22 -07:00
Yiliu Dong and OpenAI Codex
ba702e978e
[Attention] Skip sparse indexer scoring for dense short prefills ( #48407 )
...
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-28 16:17:22 +00:00
Harry Mellor and Claude Opus 5
9321aff536
[Bugfix] Wait for the linear bias before layerwise online processing ( #49805 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-07-25 17:56:32 +00:00
Harry Mellor
9a50464698
[CI] Stop flaky test from downloading model every time ( #49800 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-25 13:29:36 +00:00
Harry Mellor and Andreas Karatzas
ca0defa343
Make bare hugging_face imports forbidden ( #49726 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-25 04:18:39 -07:00
aoshen02 and aoshen02
ba18929079
[Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates ( #49178 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-07-21 18:56:39 -07:00
97a668152b
[RL Infra][FlashInfer] Enable router replay output from FlashInfer monolithic MoE kernel ( #44214 )
...
Signed-off-by: Xuanyu Zhang <xuanyu.zhang@mistral.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-20 16:45:10 -07:00
Roberto L. Castro and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7ca017778f
[Feat][Perf] Add new warmup infrastructure for JITs ( #47451 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 13:21:55 -07:00
Michael Goin and OpenAI Codex
cc25f028b7
[Loader] Improve InstantTensor loading ( #46868 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-17 16:30:02 -04:00
Joan and Dakai An
0bd6b85a1f
[Bugfix] Preserve unloaded non-persistent buffers during layerwise reload ( #44371 )
...
Signed-off-by: Joan Velja <joan.velja22@gmail.com >
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com >
2026-07-14 17:46:29 -07:00
aoshen02
3ca242d1b6
[Bugfix][R3] Exclude draft routers from expert capture ( #48622 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-07-14 17:45:35 -07:00
85c09e9885
fix: correct load_weights track logic and enable weight integrity for… ( #41811 )
...
Signed-off-by: Yipeng Hu <i26268@metax-tech.com >
Signed-off-by: HuYiPeng <144002351+MynameFelix@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Yipeng Hu <i26268@metax-tech.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-07-10 14:08:20 +00:00
Ting SUN
91b5647300
[Bugfix][Model] Allow Run:ai memory_limit sentinel values ( #47337 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-07-05 00:08:34 +00:00
