Luke Alonso and mgoin
2740c817ff
[Kernel] Add b12x FP4 MoE backend ( #52018 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-08-21 08:04:36 -07:00
Harry Mellor
f0c14b4f77
Fix weight tying ( #51665 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-20 10:31:55 +00:00
Qiming Zhang
754e1c3de4
[CI][XPU] Skip test_fused_shared_expert.py on XPU ( #53035 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-08-20 14:56:09 +08:00
b05ae5dc00
[CI][Bugfix] Complete DeepSeek-V4 FSE test fixture contract ( #52842 )
...
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com >
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-18 19:04:10 -06:00
Summer Yang
5f7a20b316
[nv] add pcp support in dsv3.2 ( #52046 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-18 14:59:08 -07:00
Woosuk Kwon and OpenAI Codex
01e56caaf2
[Bugfix][MLA] Do not use Dense MHA for GLM-5.2 ( #52512 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-18 11:20:48 -07:00
fxmarty-amd
88b2bff2c6
[MOE] Standardize and abstract fused shared expert optimization selection ( #51695 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-18 12:56:56 -04:00
Roberto L. Castro and Codex
49905ad94d
[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup ( #50174 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-08-17 10:11:14 -07:00
Misha Goin
4ab5e5012a
[Refactor] Simplify B12X linear kernels and warmup ( #52368 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-08-17 09:45:45 -07:00
Qiming Zhang and Andreas Karatzas
1f7427bc0a
[UT][XPU] fix b12x UT ( #52265 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
2026-08-14 09:49:47 -04:00
Luke Alonso
3c79b1a8bf
[Kernel] Add B12X dense linear backends ( #52016 )
...
Signed-off-by: Luke Alonso <lalonso@gmail.com >
2026-08-13 19:48:33 -07:00
aoshen02 and Codex
f2906b492e
[Bugfix] Avoid repeated layerwise reload warning scans ( #49505 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-08-12 13:53:50 +08:00
Richard Zou
cb30f6f8dd
[Testing] Fix test_sharded_state_loader ( #51736 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-08-11 21:16:09 +00:00
Andreas Karatzas and OpenAI Codex
1ab2801dde
[ROCm] Remove stale SDPA and skinny GEMM workarounds ( #50907 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 14:19:16 -04:00
6c95a641e9
[2/N][Feat][Perf] Add new warmup infrastructure for JITs. Add predicate filtering for JIT warmup, and migrate Inkling FA4 ( #49315 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 07:46:07 -04:00
aoshen02 and Roger Wang
99e62b802c
[Bugfix][Model Loader] Defer post-load attention weight processing ( #49519 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-08-10 19:16:57 -07:00
Taneem Ibrahim
9464529612
[ModelRunner v2] Enable decoder token-wise pooling ( #50931 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-06 22:21:52 +00:00
81be2e09ae
[Weight processing] Copy over new_data attributes in replace_parameter ( #49601 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:03:37 -04:00
aoshen02 and yihengz
6153dbe036
[R3] Unify routed expert shape configuration ( #50940 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: yihengz <yihengz@nvidia.com >
2026-08-04 20:20:41 -07:00
aoshen02 and aoshen02
7153fd70f6
[Bugfix][MoE] Filter packed expert weights during EP loading ( #49558 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-08-04 16:32:03 +00:00
aoshen02 and Nick Hill
42ab184ea7
[MRV2] Enable routed-experts capture ( #50721 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-03 21:00:13 +00:00
+1
4f1da84eb5
Enable gfx1250 ROCm architecture ( #46516 )
...
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Signed-off-by: Daniel <danichan@amd.com >
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com >
Signed-off-by: jpvillam <jpvillam@amd.com >
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com >
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
Co-authored-by: Daniel <danichan@amd.com >
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com >
Co-authored-by: jpvillam <jpvillam@amd.com >
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: dllehr <douglehr@amd.com >
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-31 01:41:12 +00:00
Ryan Clark and Claude Fable 5
68fb303a41
[Bugfix] Preserve Marlin runtime tensor storage across weight reload ( #48438 )
...
Signed-off-by: Ryan Clark <ryanclark2k@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-30 14:50:03 -07:00
Andreas Karatzas and OpenAI Codex
30e333ca5f
[Bugfix] Shut down private Tensorizer engines ( #49840 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-30 12:33:22 -07:00
Yiliu Dong and OpenAI Codex
ba702e978e
[Attention] Skip sparse indexer scoring for dense short prefills ( #48407 )
...
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-28 16:17:22 +00:00
Harry Mellor and Claude Opus 5
9321aff536
[Bugfix] Wait for the linear bias before layerwise online processing ( #49805 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-07-25 17:56:32 +00:00
Harry Mellor
9a50464698
[CI] Stop flaky test from downloading model every time ( #49800 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-25 13:29:36 +00:00
Harry Mellor and Andreas Karatzas
ca0defa343
Make bare hugging_face imports forbidden ( #49726 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-25 04:18:39 -07:00
aoshen02 and aoshen02
ba18929079
[Bugfix][SpecDecode] Scope MTP completeness checks outside bucketed updates ( #49178 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-07-21 18:56:39 -07:00
97a668152b
[RL Infra][FlashInfer] Enable router replay output from FlashInfer monolithic MoE kernel ( #44214 )
...
Signed-off-by: Xuanyu Zhang <xuanyu.zhang@mistral.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-20 16:45:10 -07:00
Roberto L. Castro and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7ca017778f
[Feat][Perf] Add new warmup infrastructure for JITs ( #47451 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-20 13:21:55 -07:00
Michael Goin and OpenAI Codex
cc25f028b7
[Loader] Improve InstantTensor loading ( #46868 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-07-17 16:30:02 -04:00
Joan and Dakai An
0bd6b85a1f
[Bugfix] Preserve unloaded non-persistent buffers during layerwise reload ( #44371 )
...
Signed-off-by: Joan Velja <joan.velja22@gmail.com >
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com >
2026-07-14 17:46:29 -07:00
aoshen02
3ca242d1b6
[Bugfix][R3] Exclude draft routers from expert capture ( #48622 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-07-14 17:45:35 -07:00
85c09e9885
fix: correct load_weights track logic and enable weight integrity for… ( #41811 )
...
Signed-off-by: Yipeng Hu <i26268@metax-tech.com >
Signed-off-by: HuYiPeng <144002351+MynameFelix@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Yipeng Hu <i26268@metax-tech.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-07-10 14:08:20 +00:00
Ting SUN
91b5647300
[Bugfix][Model] Allow Run:ai memory_limit sentinel values ( #47337 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-07-05 00:08:34 +00:00
Chaojun Zhang
ea9ddf59fc
[XPU][CI] Enable shared loader test ( #45977 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
2026-06-30 11:20:33 +00:00
Harry Mellor
5051698e41
Remove unnecessary load_weights methods ( #44589 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-29 01:52:23 -07:00
Ilya Markov and Markov Ilya
a2abce646f
[EPLB] Mask padding in EPLB load recording ( #38128 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Markov Ilya <markovilya19@gmail.com >
Co-authored-by: Markov Ilya <markovilya19@gmail.com >
2026-06-28 19:43:58 -07:00
44d95069e9
Enable DeepSeek V4 and GLM-5.1 on SM120 ( #43477 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-06-22 11:54:14 -07:00
Ting SUN
9d4b87f4f0
[Bugfix][Model] Validate DefaultModelLoader / LoadConfig and fail with clear errors ( #45196 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-06-17 21:46:33 +00:00
405c7cf283
[ZenCPU] Add zencpu Platform Runtime Logging and Docs ( #42726 )
...
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com >
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com >
2026-06-16 08:23:12 -04:00
9d808e2309
[Core] Use fastsafetensors ParallelLoader for weight loading ( #40183 )
...
Signed-off-by: Git Bisector <gitbisector@gmail.com >
Signed-off-by: gitbisector <gitbisector@gmail.com >
Signed-off-by: git bisector <gitbisector@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-06-15 22:32:05 -07:00
Ting SUN
3d6ce816f0
[Bugfix][Model] Validate runai_streamer model_loader_extra_config ( #45291 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-06-14 19:23:30 -07:00
Ting SUN
c1076839c9
[Bugfix][Model] Pass revision by name in Run:ai and bitsandbytes index downloads ( #45308 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-06-11 20:21:46 -07:00
jpwang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
6f573f486b
[Bugfix] Initialize missing attributes in mistral eagle ( #45217 )
...
Signed-off-by: jpwang <jpwang@smail.nju.edu.cn >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-12 08:21:01 +08:00
c9e5bf8135
[Bugfix] Fix layerwise reload dropping params after a composed weight loader ( #44814 )
...
Signed-off-by: hallerite <git@hallerite.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com >
2026-06-10 06:42:05 -07:00
bnellnm
dc68bd8c41
[MoE Refactor] FusedMoE/MoERunner inversion refactor ( #41184 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-06-08 10:42:58 -04:00
Taneem Ibrahim
eafbb06331
[Misc] Replaced asserts with proper exceptions to improve UX for pooling ( #44593 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-06-06 05:57:26 +00:00
Harry Mellor and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
62215e72c6
Remove KV cache scale boilerplate from model weight loading methods ( #43167 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-05 05:19:04 -07:00
