 Sumanth R Hegdeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
3cb83c9592
|
Add model to WeightTransferEngine.__init__ (#42922)
Signed-off-by: SumanthRH <sumanthrh99@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-22 17:52:15 -07:00 |
|
|
Nick Hill
|
47d4407d7c
|
[Model Runner V2] Support sharing kv cache layers (#35045)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-22 22:18:23 +00:00 |
|
|
Isotr0py
|
f0feb15e7f
|
[Multimodal] Simplify ViT CUDA graph interfaces (#41234)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-05-22 22:31:00 +08:00 |
|
|
Li, Jiang
|
65b7a812a2
|
[CPU] Experimentally enable Triton and MRV2 (#43225)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-05-22 01:48:17 -07:00 |
|
|
haosdentandWentao Ye
|
5ea76fa89a
|
[CI] Fix test_lora_with_spec_decode on V2 model runner (#43314)
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-05-22 14:24:18 +08:00 |
|
|
Lanze Liu
|
39d5fa96a7
|
[Bugfix] Zero stale is_prefilling in padded CUDA graph rows for Mamba (#41873)
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
|
2026-05-21 15:42:42 -07:00 |
|
|
Francesco Fuscoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
b730c46352
|
[Perf] [Hybrid] Fused Triton kernel for GPU-side Mamba state postprocessing (#40172)
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-21 04:50:54 -07:00 |
|
|
Wentao YeandNick Hill
|
9640970de2
|
[Model Runner V2] Fix lora Triton Error [CUDA]: device-side assert triggered (#43139)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-21 01:00:30 +00:00 |
|
|
Kebe
|
19cf334207
|
[Feature] Support manually enabling the cumem allocator (#33648)
Signed-off-by: Kebe <mail@kebe7jun.com>
|
2026-05-20 08:58:30 -04:00 |
|
|
Izik Golan
|
2ae910ed88
|
[Perf] Avoid forward scan for async output placeholders (#42938)
|
2026-05-19 20:16:07 -07:00 |
|
|
Benjamin Chislett
|
c628a93a64
|
[Perf][Bugfix] Update dflash aux layer indexing (#40727)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-05-19 20:15:57 -07:00 |
|
|
Nick Hill
|
39bba710be
|
[MRV2][BugFix] Fix default-stream CG capture in P/W LoRA case (#43160)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-19 19:19:05 -07:00 |
|
|
Doğaç Eldenk
|
1242196295
|
[Model] Support post-norm architecture for EAGLE-3 supeculators (#42764)
Signed-off-by: Doğaç Eldenk <dogacel@gmail.com>
|
2026-05-19 13:39:00 -07:00 |
|
|
Nick Hill
|
b82e908b4c
|
[Perf][4/n] Eliminate various GPU<->CPU syncs (#42347)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-19 10:35:54 -04:00 |
|
|
Nicolò LucchesiandNick Hill
|
fba010dd74
|
[Bugfix][MRV2] Fix KVCache tensor explicit kernel_block_size dim (#42766)
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-18 20:25:41 -07:00 |
|
|
Ofir Zafrir
|
8f16c4a5c0
|
[BugFix][CPU][Spec Decode] Fix Eagle implementation on CPU backend (#42468)
Signed-off-by: Ofir Zafrir <ofir.zafrir@intel.com>
|
2026-05-19 03:16:07 +00:00 |
|
|
Wentao Ye
|
e26736973a
|
[Model Runner V2] Fix prompt logprobs calculation Sizes of tensors must match error (#42778)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-18 08:27:21 -07:00 |
|
|
Michael GoinandClaude Opus 4.7
|
f5d3dc7115
|
[Model Runner v2] Support update_config (#42783)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-18 10:26:07 -04:00 |
|
|
roikoren755
|
737bfa3a43
|
[Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash (#41233)
Signed-off-by: Roi Koren <roik@nvidia.com>
|
2026-05-18 14:54:00 +03:00 |
|
|
Soyaazz
|
990f49bdcb
|
[MM][CG] Enable encoder Cudagraph for Step3VL (#42224)
Signed-off-by: JisoLya <523420504@qq.com>
Signed-off-by: Soyaazz <523420504@qq.com>
|
2026-05-17 20:19:13 -07:00 |
|
|
liuzhenwei
|
ff712f6447
|
[MRV2][XPU] add Model Runner V2 log (#42710)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
|
2026-05-17 04:15:50 +00:00 |
|
|
Jiangyun Zhu
|
8a56da3845
|
[Experimental] Breakable CUDA graph (#42304)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-05-16 22:04:12 +08:00 |
|
|
Keyi Li
|
d0921bafef
|
[Bugfix] Unwrap VLM wrappers for EPLB on Model Runner V2 (#42706)
|
2026-05-16 07:20:33 +08:00 |
|
|
Wentao Yeandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
6147c70224
|
[Model Runner v2] Support reload weights (sleep mode) (#42673)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-15 16:41:23 +00:00 |
|
|
PatchyTISandPatchouliTaisa
|
0162596603
|
[Model Runner V2] FP32 gumbel sampling. (#41775)
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
|
2026-05-15 09:20:08 -07:00 |
|
|
BadrBasowid
|
fb5bd03f51
|
[Perf] Set IR Op Priority Once at Worker Init (#42631)
Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
|
2026-05-15 15:56:13 +00:00 |
|
|
Wentao Ye
|
af9616d845
|
[Model Runner V2] Fix kv_connector pre_forward order (#42676)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-15 08:13:59 -07:00 |
|
|
Li, Jiang
|
95cfe102a5
|
[Bugfix] Ensure embeding model compilation on CPU (#42709)
Signed-off-by: jiang1.li <jiang1.li@intel.com>
|
2026-05-15 18:58:19 +08:00 |
|
|
Daniel WandLi, Jiang <jiang1.li@intel.com>
|
27b85d2084
|
[Bugfix] Clarify CPU backend memory error messages reference shared flag (#42479)
Signed-off-by: daniel-devlab <282598346+daniel-devlab@users.noreply.github.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-05-15 06:35:05 +00:00 |
|
|
|
bf610c2f56
|
[Bugfix] Fix inverted condition causing thinking_token_budget to be silently ignored (#41674)
Signed-off-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: Keyi Li <likey6688@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-15 12:48:49 +08:00 |
|
|
Giancarlo Delfin
|
3b6a204789
|
[Model Runner V2][Bug Fix][DSV4] Ensure lazy attention state initializations happen during cudagraph capture (#42444)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-05-14 16:16:17 -07:00 |
|
|
Wentao YeandNick Hill
|
ae4f59f0ec
|
[Model Runner v2] Oracle for model runner v2 - qwen3 dense model by default [1/N] (#39337)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-14 10:02:33 -07:00 |
|
|
Baorun (Lauren) Mu
|
a7737cb4f3
|
[Fix] Misc Fixes in ViT CUDA Graph (#38040)
Signed-off-by: Baorun Mu <bmu@nvidia.com>
|
2026-05-14 23:49:06 +08:00 |
|
|
|
c7560af424
|
[RFC] Replace shared-memory routed experts with ModelRunnerOutput transfer and HTTP support (#39568)
Signed-off-by: xhx1022 <1737006628@qq.com>
Signed-off-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: arlenxu <arlenxu@tencent.com>
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com>
|
2026-05-14 14:12:30 +00:00 |
|
|
aoshen02
|
8c79ad6580
|
Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
2026-05-13 23:49:01 -07:00 |
|
|
Roy Wang
|
77e1421a68
|
[Bugfix] Fix EPLB initialization for VLM wrapper models (#39805)
Signed-off-by: esmeetu <jasonailu87@gmail.com>
|
2026-05-14 02:26:15 +00:00 |
|
|
Siddharth Bedekar
|
f51f6844f9
|
[Bugfix][Spec Decode] Wire draft_probs into probabilistic draft_model rejection (#40269)
|
2026-05-13 21:04:03 -04:00 |
|
|
Yufeng HeandMatthew Bonanni
|
b2198670b1
|
[Bugfix] V1: support tuple model outputs in ubatch wrapper (dbo + spec decode) (#40789)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-05-13 15:47:51 -07:00 |
|
|
Nick Hill
|
a505cf807e
|
[ModelRunner V2] Share identical MTP weights (#42538)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-13 18:57:04 +00:00 |
|
|
Michael GoinandClaude Opus 4.6
|
2f821faeae
|
[Spec Decode] Support hybrid attention models in extract_hidden_states (#39949)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-05-13 10:45:53 -07:00 |
|
|
CynicDora
|
256dbcaabf
|
[Feature] Support custom callable proposer backend for speculative decoding (#39487)
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
|
2026-05-13 16:53:01 +00:00 |
|
|
Giancarlo Delfin
|
fe5b4e0fe7
|
[Model Runner V2] Apply synthetic mode to probabilistic rejection sampler (#41035)
|
2026-05-12 13:37:03 -07:00 |
|
|
liuzhenweiandKunshang Ji
|
8517cdaf90
|
[XPU] update dp rank w/o env-var isolation (#39856)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-05-12 14:49:54 +08:00 |
|
|
Lucas Kabela
|
4e498b5e5c
|
[Bugfix][Performance Improvement] Improve penalties triton kernel performance (#40657)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-05-12 05:47:20 +00:00 |
|
|
|
d7af6b34d8
|
[Model Runner V2] Bug fix: logprob dtype int64/int32 issue (#41761)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 21:55:43 +00:00 |
|
|
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bbee532988
|
[Perf][1/n] Eliminate various GPU<->CPU syncs (#41429)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 20:36:03 +00:00 |
|
|
Nick Hill
|
9af6a5ed75
|
[Model Runner V2] Fix seq_lens_cpu_upper_bound (#42202)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-11 10:37:50 -07:00 |
|
|
Wang XingranandHongjian Zhang
|
0b272a6e01
|
[Bugfix] Fix SP pass for multimodal models and PP+SP residual handling (#33322)
Signed-off-by: Xingran Wang <wangxingran123456@outlook.com>
Signed-off-by: Hongjian Zhang <hirokenovo@gmail.com>
Co-authored-by: Hongjian Zhang <hirokenovo@gmail.com>
|
2026-05-09 19:44:16 -07:00 |
|
|
aoshen02
|
006af4b956
|
[Bugfix] Skip routed-experts hot path when disabled (#42148)
|
2026-05-09 18:01:04 -07:00 |
|
|
Sumanth R Hegde
|
e3b65a5ba0
|
[feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212)
|
2026-05-08 18:03:33 -07:00 |
|