|
01af92e175
|
[Feature][Model Runner V2] Support extract_hidden_states speculation (#49811)
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-20 11:13:43 -04:00 |
|
|
|
a1c5b1fd9f
|
[Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701)
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 15:12:43 +08:00 |
|
|
Giancarlo Delfin
|
d4f4d3f40f
|
[Model Runner V2][Spec Decode] Fix draft logits cache column stride in gumbel_sample (#53017)
|
2026-08-19 17:39:30 -07:00 |
|
|
Willow Lopez
|
f936a267f9
|
[Bugfix][XPU] Fix Mamba state pointer overflow (#48109)
Signed-off-by: Oxygen56 <jiangth99@163.com>
|
2026-08-19 10:45:41 +08:00 |
|
|
Itay Alroy
|
a2257f95b7
|
[Elastic EP] Reduce eager-mode reconfiguration downtime (#51885)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-08-19 02:13:06 +00:00 |
|
|
|
0db502c8d8
|
[Kimi-K3] support DCP partial prefix cache hit (#50493)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-17 17:05:51 -07:00 |
|
|
Canlin GuoandNick Hill
|
455edc022b
|
[ModelRunnerV2] Support prompt embeds (#42963)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-17 21:58:50 +00:00 |
|
|
Roberto L. CastroandCodex
|
49905ad94d
|
[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup (#50174)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-17 10:11:14 -07:00 |
|
|
|
70afdedc10
|
[K3] support recoverssm for K3 (#51855)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-17 19:34:14 +08:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
a02cfccbc6
|
[Bugfix][Mamba] Fix overlapping state copy race (#50729)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-17 08:29:41 +02:00 |
|
|
Cheng RuiandRui Rui4 Cheng
|
6664d397bf
|
[Performance][MRV2] Cache logits-processing request state (#52329)
Signed-off-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
Co-authored-by: Rui Rui4 Cheng <chengrui4@lenovo.com>
|
2026-08-17 02:36:41 +00:00 |
|
|
|
1b079c40ff
|
[Bugfix][Model Runner V2][Spec Decode] Fix off-by-one in bad_words draft-prefix matching (#52311)
Signed-off-by: jyan <r_02213@sjtu.edu.cn>
Co-authored-by: jyan <r_02213@sjtu.edu.cn>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-16 09:02:20 +00:00 |
|
|
|
fa9d67f782
|
[EC Connector] Added Build Connector Worker Meta for EC Connector (#49585)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-08-15 15:40:38 -07:00 |
|
|
Gabriel Wu
|
97388c44f9
|
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538)
|
2026-08-15 07:01:32 -07:00 |
|
|
James E T SmithandAndreas Karatzas
|
9b0ab5dd53
|
[ROCm][AMD] Enable preshuffled sparse indexing for 16-token blocks (#51216)
Signed-off-by: jamesETsmith <james.smith9113@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-08-14 12:38:14 -05:00 |
|
|
Lucas WilkinsonandClaude
|
57bd0ed441
|
[5/N][KV-Cache Layout Refactor] Backend-published KV packing via customize_spec (#51704)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-14 09:47:55 -04:00 |
|
|
|
95c9144424
|
[LoRA][Gemma4] Support vision tower LoRA (#42662)
Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <joy25810@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: linitra24 <joy25810@gmail.com>
|
2026-08-13 22:07:43 +08:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
79f3183f86
|
[Bugfix] Bound KV block zeroing launch geometry (#52058)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 03:42:26 +00:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
|
0a94d85a66
|
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
|
2026-08-11 17:15:06 -07:00 |
|
|
鐘天楽
|
a367cdbca0
|
[Profiler] Add minimal Triton Proton profiling backend (#48789)
Signed-off-by: Luossu <zhluosuu@outlook.com>
|
2026-08-11 14:51:57 -07:00 |
|
|
stefankoncarevic
|
3e372c5ff2
|
[Bugfix][ROCm] Give KV-first attention blocks their own page in hybrid models (#51837)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
|
2026-08-11 16:52:30 -04:00 |
|
|
Andreas Karatzas
|
cc668c5b72
|
[CI][Bugfix][V1] Remove stale FlashAttention metadata arguments (#51854)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-08-11 18:54:56 +00:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
1ab2801dde
|
[ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-11 14:19:16 -04:00 |
|
|
Michael Goinandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
e3fe212eaf
|
[Bugfix] Generalize KV block zeroing to AttentionSpec (#51749)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-11 12:13:09 -04:00 |
|
|
Guan-Ming Chiuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
457a5f34db
|
[Bugfix][MRV2] Support encoder timing stats in model runner V2 (#50020)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-12 00:00:49 +08:00 |
|
|
Yizhou
|
a311916a29
|
[MRV2][Spec] Fuse AR speculator multi-step decodes back into one CUDA graph (#46849)
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
|
2026-08-11 15:06:22 +08:00 |
|
|
|
d608dfabfd
|
[Bugfix][MRV2] Reserve spec-decode lookahead blocks in V2 warmup (#51438)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
|
2026-08-08 23:58:43 +00:00 |
|
|
coltonottleyandColton Ottley
|
0df620d429
|
[Test] Add packed DeepSeek-V4 KV zeroer geometry regression (#51288)
Signed-off-by: Colton Ottley <colton@ottleyengineering.com>
Co-authored-by: Colton Ottley <colton@ottleyengineering.com>
|
2026-08-07 12:11:09 -07:00 |
|
|
Julien Debache
|
ae934ba8a5
|
feat: extended EPLB support for Mistral Large 3 and additional MoE backends (#48355)
Signed-off-by: jdebache <jdebache@nvidia.com>
|
2026-08-07 05:32:52 -07:00 |
|
|
Tianyu Guo
|
dd856e48bb
|
[Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance (#51222)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-08-06 20:40:47 -07:00 |
|
|
ChaunceyandNick Hill
|
72c0d67657
|
[Feat] Support thinking_token_budget in Model Runner V2 (#46727)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-07 03:36:18 +00:00 |
|
|
wangxian001andNick Hill
|
d6af803f43
|
[Bugfix] Fix packed KV block zeroing stride (#50276)
Signed-off-by: wangxian001 <120719093+wangxian001@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-06 13:49:06 -07:00 |
|
|
Giancarlo Delfin
|
81bc196913
|
[Model Runner V2] Cache draft logits in model's LM head dtype (#50910)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-05 19:12:46 -07:00 |
|
|
|
b50fdebce0
|
[Bugfix] Fix level-2 sleep/wake/reload with enable_lora=True (#39935)
Signed-off-by: Silen Naihin <silen.naihin@gmail.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
|
2026-08-06 09:13:10 +08:00 |
|
|
bnellnm
|
373fe8b83e
|
[MoE Refactor] Remove MoE legacy code (#51078)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-08-05 14:35:36 -07:00 |
|
|
Tyler Michael SmithandOpenAI Codex
|
12292d94b2
|
[CI] Detect and fail evals on when NaNs appear in logits (#50323)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-04 23:02:58 +00:00 |
|
|
Varun Vinayak ShenoyandNick Hill
|
e578de311c
|
[ModelRunnerV2] Fix scalar Mamba state update with int32 mappings (#50327)
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-03 22:05:12 +00:00 |
|
|
Raphaël Rialland
|
4a3447d200
|
[Bugfix][Model Runner V2] Restore multimodal draft capability detection (#50417)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
|
2026-08-03 14:18:14 -07:00 |
|
|
aoshen02andNick Hill
|
42ab184ea7
|
[MRV2] Enable routed-experts capture (#50721)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-03 21:00:13 +00:00 |
|
|
Francesco FuscoandClaude
|
755513d9a2
|
[Hybrid] Stage the postprocess inputs with a single loop over the request list (#48120)
Signed-off-by: Francesco Fusco <ffu@zurich.ibm.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-03 20:24:22 +00:00 |
|
|
Matthew Bonanni
|
a0cd2b69b3
|
[Bugfix] Universally align block table width to 128 tokens (#50302)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-31 08:52:17 -07:00 |
|
|
Nick HillandJeff Ma
|
6fbbcf2151
|
[BugFix] Stop dummy runs from writing mamba state through stale block-table rows (#49757)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Jeff Ma <jeffjma@umich.edu>
|
2026-07-29 01:09:41 +00:00 |
|
|
Kevin Glynn
|
56f31af62a
|
[Bugfix] Fix /wake_up crash on hybrid models (Mamba/DeltaNet) (#41602)
Signed-off-by: Kevin Glynn <kevglynn@gmail.com>
|
2026-07-29 00:17:35 +00:00 |
|
|
Nick Hill
|
0d0504b54c
|
[Core] Warm up runner-owned Triton kernels before the first request (#49903)
|
2026-07-28 07:58:02 -07:00 |
|
|
liuzhenwei
|
afc94523c9
|
[XPU][CI] Use platform device in InputBatch V2 test (#49939)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
|
2026-07-27 15:28:02 +08:00 |
|
|
Nick Hill
|
50aa830482
|
[BugFix][MRV2] Don't create dummy requests longer than max_model_len (#49751)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-27 02:04:20 +00:00 |
|
|
Elvir CrnčevićandClaude Opus 4.6
|
972848f276
|
[Bugfix] Support non-uniform page sizes in KVBlockZeroer (#49704)
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-07-24 13:38:59 -07:00 |
|
|
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
7b40fb9645
|
[UX] Reject incompatible nested runtime overrides (#49247)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-24 07:16:22 -07:00 |
|
|
|
833483f357
|
Encoder cache extension hooks (#48218)
Signed-off-by: hotTea <958436561@qq.com>
Signed-off-by: hanxi-java <634498162@qq.com>
Co-authored-by: hanxi-java <634498162@qq.com>
Co-authored-by: 韩熙 <63780107+hanxi-java@users.noreply.github.com>
|
2026-07-24 02:52:00 -07:00 |
|