 Jiangyun Zhu
|
98ed0856f3
|
[Model] add GLM-5.3-Flash support (#53906)
|
2026-09-03 09:40:35 -07:00 |
|
|
Andrii SkliarandAndrii Skliar
|
adebc41b7e
|
[Mamba] Add FlashInfer ReplaySSM backend (#52506)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-09-01 10:26:43 -04:00 |
|
|
Roberto L. CastroandCodex
|
e16b5e518d
|
[1/N][warmup][DSv4] Migrate generic MLA metadata and indexing kernels (#50175)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-31 23:26:05 -04:00 |
|
|
Nick Hill
|
c28feab989
|
[Core][MRV2] Freeze gc during V2 CG capture; skip per-descriptor cleanup (#54646)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-31 19:37:46 -07:00 |
|
|
AusarandClaude
|
6bafc049aa
|
[Bugfix][PP] Never drop a decoding request from the sampled-token broadcast (#54436)
Signed-off-by: ArcheyChen <45146080+ArcheyChen@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-31 23:49:54 +00:00 |
|
|
+5
|
e126687a9a
|
[Model] Support Qwen3.8-Flash-Next (#53896)
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: andyluo7 <andy.luo@amd.com>
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: KungYork <firelikest@163.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-31 05:57:56 +00:00 |
|
|
Nicolò Lucchesi
|
7f4793eaa3
|
[Nixl][PD] DCP support for MLA models (#50611)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-29 16:03:53 +00:00 |
|
|
Nick HillandKimi Code
|
96242aa50d
|
[Bugfix][MRV2] Release layer-bound KV cache memory in shutdown() (#54246)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
|
2026-08-28 21:27:09 +00:00 |
|
|
Ronaldandaoshen02
|
479eeb32d2
|
[Bugfix][MRV2] Isolate sleep-mode KV allocations (#53508)
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-08-27 15:08:28 -07:00 |
|
|
Qiu Chunshuo
|
b3af042abd
|
Bugfix: use PCP slot mappings for PIECEWISE capture (#53869)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
|
2026-08-27 05:39:36 +00:00 |
|
|
Giancarlo Delfin
|
5acc1c4e4b
|
[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill (#53694)
|
2026-08-26 20:59:35 -07:00 |
|
|
|
b1fbbc2ade
|
BugFix(PCP): use persistent input buffers for PIECEWISE CUDA graphs (#53515)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-26 05:58:13 +00:00 |
|
|
Misha Goin
|
7de96050c5
|
[Spec Decode] Enable adaptive DSpark on SM100 sparse MLA (#52783)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-25 13:31:15 -04:00 |
|
|
Giancarlo Delfin
|
d154d90d6c
|
[Model Runner V2] batch-sharded sample (#50465)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-24 15:49:57 -07:00 |
|
|
|
a4d70bef37
|
[Model Runner V2] Reserve CUDA graph memory (#53306)
Signed-off-by: Anh Tran <anh.tran.3889@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Anh Tran <anh.tran.3889@gmail.com>
Co-authored-by: anhtra3889 <anhtra@nvidia.com>
|
2026-08-23 23:11:19 -07:00 |
|
|
Tianyu Guo
|
7ca49fbe4b
|
[Refactor][Model Runner V2][Multimodal] Move the encoder-only path out of the shared runner (#53176)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-08-22 05:11:44 +00:00 |
|
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
|
|
01af92e175
|
[Feature][Model Runner V2] Support extract_hidden_states speculation (#49811)
Signed-off-by: Zupeng Wang <71580390+zupengwang@users.noreply.github.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI <support@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-20 11:13:43 -04:00 |
|
|
|
a1c5b1fd9f
|
[Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701)
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 15:12:43 +08:00 |
|
|
Itay Alroy
|
a2257f95b7
|
[Elastic EP] Reduce eager-mode reconfiguration downtime (#51885)
Signed-off-by: Itay Alroy <ialroy@nvidia.com>
|
2026-08-19 02:13:06 +00:00 |
|
|
|
0db502c8d8
|
[Kimi-K3] support DCP partial prefix cache hit (#50493)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
|
2026-08-17 17:05:51 -07:00 |
|
|
Canlin GuoandNick Hill
|
455edc022b
|
[ModelRunnerV2] Support prompt embeds (#42963)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-17 21:58:50 +00:00 |
|
|
Roberto L. CastroandCodex
|
49905ad94d
|
[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup (#50174)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-08-17 10:11:14 -07:00 |
|
|
|
4d2a68d64d
|
[Bugfix][Spec Decode][Structured Output] DSpark: fix the grammar bitmask mapping when the draft budget is zero (#52436)
Signed-off-by: oops-oom <73481342@qq.com>
Co-authored-by: oops-oom <73481342@qq.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-08-16 11:09:23 +00:00 |
|
|
|
fa9d67f782
|
[EC Connector] Added Build Connector Worker Meta for EC Connector (#49585)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-08-15 15:40:38 -07:00 |
|
|
Gabriel Wu
|
97388c44f9
|
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538)
|
2026-08-15 07:01:32 -07:00 |
|
|
Nick Hill
|
d6f17f3d53
|
[MRV2] Support attention-free models (#52374)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-14 23:54:50 +00:00 |
|
|
Isotr0pyandOpenAI Codex
|
66728feb1f
|
[MRV2][Multimodal] Enable encoder cuda graph for model runner v2 (#49852)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-14 09:33:21 +00:00 |
|
|
|
50ba4bc6b2
|
[Feature] Mask Replay (#49577)
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 08:44:31 +00:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
02ac17851d
|
[Refactor][MRV2] Unify uniform decode token count helper (#51917)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-12 05:58:43 +00:00 |
|
|
|
0a94d85a66
|
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
|
2026-08-11 17:15:06 -07:00 |
|
|
Guan-Ming Chiuandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
457a5f34db
|
[Bugfix][MRV2] Support encoder timing stats in model runner V2 (#50020)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-12 00:00:49 +08:00 |
|
|
Wentao Ye
|
a671679e9f
|
[MRv2 Feature] MR v2 weight offloading support (#51413)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-08-07 13:20:21 -06:00 |
|
|
Tianyu Guo
|
dd856e48bb
|
[Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance (#51222)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
|
2026-08-06 20:40:47 -07:00 |
|
|
ChaunceyandNick Hill
|
72c0d67657
|
[Feat] Support thinking_token_budget in Model Runner V2 (#46727)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-07 03:36:18 +00:00 |
|
|
Taneem Ibrahim
|
9464529612
|
[ModelRunner v2] Enable decoder token-wise pooling (#50931)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-08-06 22:21:52 +00:00 |
|
|
Nick Hill
|
c5d470ac4c
|
[ModelRunner V2] Minor indexing optimizations (#51210)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-06 14:44:13 -07:00 |
|
|
QiuandLucas Wilkinson
|
811622c410
|
[Refactor][PCP] Make PCPManager construction extensible (#50066)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-08-05 23:05:14 +00:00 |
|
|
bnellnm
|
373fe8b83e
|
[MoE Refactor] Remove MoE legacy code (#51078)
Signed-off-by: Bill Nell <bnell@redhat.com>
|
2026-08-05 14:35:36 -07:00 |
|
|
Wentao Ye
|
4f819f801b
|
[Model Runner v2] E/P/D disaggregation support (#38390)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-08-04 15:52:23 +00:00 |
|
|
aoshen02andNick Hill
|
42ab184ea7
|
[MRV2] Enable routed-experts capture (#50721)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-03 21:00:13 +00:00 |
|
|
Taneem Ibrahim
|
652ba59229
|
[Model Runner V2] Enable encoder token embedding (#50574)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-08-01 07:55:36 -04:00 |
|
|
Michael GoinandOpenAI Codex
|
90329913e9
|
[UX] Reduce startup log noise (#50590)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-07-31 13:04:34 -07:00 |
|
|
Matthew Bonanni
|
a0cd2b69b3
|
[Bugfix] Universally align block table width to 128 tokens (#50302)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-31 08:52:17 -07:00 |
|
|
Taneem Ibrahim
|
0f17394564
|
[Model Runner V2] Enable encoder token classification (#50293)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-07-31 04:15:36 +00:00 |
|
|
Giancarlo Delfin
|
dec13a33b7
|
[Model Runner V2][Spec Decode] Add multi-layer MTP speculator (#48892)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-07-30 15:54:15 -07:00 |
|
|
Taneem Ibrahim
|
43eaefba5a
|
[ModelRunner V2] Enable sequence pooling for embedding and classification models (#48791)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-07-29 06:30:45 -07:00 |
|
|
Nick HillandJeff Ma
|
6fbbcf2151
|
[BugFix] Stop dummy runs from writing mamba state through stale block-table rows (#49757)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Jeff Ma <jeffjma@umich.edu>
|
2026-07-29 01:09:41 +00:00 |
|
|
Wentao Ye
|
b5bcb3ce88
|
[Refactor] Remove dead code in multiple files (#49745)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-27 15:58:26 -04:00 |
|