 Jiangyun Zhu
|
98ed0856f3
|
[Model] add GLM-5.3-Flash support (#53906)
|
2026-09-03 09:40:35 -07:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
e52407ef4c
|
[ROCm][CI] Add MTP and other spec-decode acceptance coverage (#53399)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-02 03:27:05 -05:00 |
|
|
Zihan ZhangandClaude Opus 5
|
d61b6e1878
|
[Bugfix][Spec Decode] Take the DFlash draft's RoPE layout from its own config (#54373)
Signed-off-by: SubSir <tiancaizhangdaxian@sjtu.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-31 18:13:20 -04:00 |
|
|
fanxingran
|
080a66a69c
|
[Bugfix][ROCm] Capture CUDA graphs on the current stream (#53818)
Signed-off-by: fanxingran <xingran.fan@amd.com>
|
2026-08-26 23:16:45 +08:00 |
|
|
danisereb
|
a7195188a4
|
Add MTP support for Nemotron VL models (#53121)
Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
|
2026-08-24 00:59:56 +00:00 |
|
|
Woosuk Kwon
|
b09bd69b5b
|
[Model][NVIDIA] Route DSA models to the CUDA non-compiled path (#52861)
|
2026-08-19 00:13:52 -07:00 |
|
|
|
6adad08767
|
Add Muse Glimmer model support (#51655)
Signed-off-by: Tiezhen Wang <tiezhen@inferact.ai>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Beto de Paola <betodepaola@meta.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: zixi-qi <zixi@inferact.ai>
|
2026-08-14 12:28:56 +08:00 |
|
|
wangxiyuanandBenjamin Chislett
|
355a338b8f
|
[BugFix][SpecDecode] Fix dspark parallel_drafting_token_id init bug (#51602)
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-08-10 13:33:10 -04:00 |
|
|
Nick Hill
|
9b0afeb4f6
|
[Perf] Avoid some more unnecessary GPU<->CPU syncs (#51458)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-08 17:13:26 -07:00 |
|
|
Raphaël Rialland
|
4a3447d200
|
[Bugfix][Model Runner V2] Restore multimodal draft capability detection (#50417)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
|
2026-08-03 14:18:14 -07:00 |
|
|
+13
|
aeeb36b1f1
|
[New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-07-30 03:49:58 -07:00 |
|
|
|
32e657e689
|
[BugFix] eagle draft max position embeddings (#49343)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-29 01:24:47 -07:00 |
|
|
Siddhant BhartiandClaude Fable 5
|
0b6aa3c47c
|
[Bugfix][Spec Decode] Size DFlash query buffers for cudagraph-padded batches (#50065)
Signed-off-by: siddhant-bharti <sbharti@together.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-28 19:34:22 +00:00 |
|
|
Giancarlo Delfin
|
272abd5f48
|
[Tests][Spec Decode] Add gemma4 MTP acceptance rates test (#47920)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-07-27 16:15:54 -07:00 |
|
|
Woosuk Kwon
|
213f681f81
|
Revert "[Perf][GLM-5.2] Blackwell decode optimizations" (#49768)
|
2026-07-24 16:04:59 -07:00 |
|
|
|
dd72658e7d
|
[Perf][GLM-5.2] Blackwell decode optimizations (#48597)
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com>
Signed-off-by: zhou <zhou@zhoudeMacBook-Pro.local>
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: zhou <zhou@zhoudeMacBook-Pro.local>
Co-authored-by: stefankoncarevic <Stefan.Koncarevic@amd.com>
|
2026-07-23 21:36:27 -07:00 |
|
|
Evan TakahashiandMichael Goin
|
b2b8f679d0
|
[Bugfix][Spec Decode] Restrict embedding-width share guard to EAGLE drafts (#47953)
Signed-off-by: Evan Takahashi <evan10takahashi@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-07-21 10:53:37 -07:00 |
|
|
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f17be06fbe
|
[Perf] Optimize clamp to clamp_ (#48143)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-16 18:41:07 -04:00 |
|
|
Michael GoinandClaude Opus 4.8
|
8bfd683901
|
[Spec Decode] Add kv_cache_dtype to speculative_config to control separately from target (#48787)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-15 23:49:23 -06:00 |
|
|
Michael GoinandClaude Fable 5
|
ecf4aa5ce2
|
[Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell (#48167)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-15 12:44:01 -07:00 |
|
|
Kunshang JiandHarry Mellor
|
07f9baf756
|
Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-06 14:18:33 +01:00 |
|
|
|
d2ec433e37
|
[XPU] Fix Eagle3 initialization on XPU (#43957)
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-06 08:46:05 +08:00 |
|
|
adamkbaranowski
|
4c3c64fcf7
|
Add Laguna XS.2.1 DFlash drafter support (#46853)
Signed-off-by: Adam Baranowski <adam.baranowski@poolside.ai>
|
2026-07-02 18:09:27 -07:00 |
|
|
|
3af8789559
|
[Feature] Universal speculative decoding for heterogeneous vocabularies (TLI) (#38174)
Signed-off-by: wan-danfeng <wandanfeng0802@gmail.com>
Signed-off-by: Wonderful <wandanfeng0802@gmail.com>
Co-authored-by: Wan_DF <wonderful199082@126.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-07-02 01:34:20 -07:00 |
|
|
Giancarlo Delfin
|
d63c8e9444
|
[BugFix][Spec Decode] Compact shared topk indices buffer after first MTP draft step (#47238)
|
2026-07-01 21:38:51 -07:00 |
|
|
Kunshang JiandHarry Mellor
|
e840f0d3f5
|
[Platform] Replace torch.cuda.Event with torch.Event (#47140)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-30 08:39:59 -07:00 |
|
|
Ilya MarkovandMarkov Ilya
|
a2abce646f
|
[EPLB] Mask padding in EPLB load recording (#38128)
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
|
2026-06-28 19:43:58 -07:00 |
|
|
guybdandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2396d91e93
|
[CPU][Spec Decode] Enable DFlash SD for CPU (#44029)
Signed-off-by: guybd <guy.boudoukh@intel.com>
Signed-off-by: Guy Boudoukh <guy.boudoukh@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-25 15:32:48 +08:00 |
|
|
Benjamin Chislett
|
4c5bc41ba6
|
[Bugfix][Spec Decode] Fix probabilistic sampling for parallel drafting (#45956)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-06-24 05:36:23 +00:00 |
|
|
Benjamin Chislett
|
89bd2c14d3
|
[Spec Decode] Add Qwen3 architecture support for EAGLE3 (#43132)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-06-21 13:55:26 -07:00 |
|
|
Nick Hill
|
7df3d7dada
|
[Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-20 20:02:24 -07:00 |
|
|
Jared Wen
|
ab66606993
|
[bugfix]Indexer init skip and MTP TopK share for iteration (#45895)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
|
2026-06-19 09:57:51 +08:00 |
|
|
Jonathan MamouandLi, Jiang <jiang1.li@intel.com>
|
351c72d6e5
|
[CPU] Skip Triton kernel monkey-patches when Triton-CPU is available (#44991)
Signed-off-by: jmamou <jonathan.mamou@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-06-18 18:59:30 +08:00 |
|
|
|
0a1c5034f5
|
[Model] Add MiniMax M3 support (#45381)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-06-16 01:01:25 +08:00 |
|
|
|
4ef4492e9b
|
[V1][Spec Decode] Add Dynamic SD (#32374)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-06-14 00:14:27 -07:00 |
|
|
|
eb28452b10
|
[Model] Add DiffusionGemma Support (#45163)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
|
2026-06-11 22:17:35 -07:00 |
|
|
yiheng
|
bd2d83ff31
|
[SpecDecode] Reduce TP communication for large-vocab draft models speculative decoding (#39419)
Signed-off-by: EanWang211123 <wangyiheng@sangfor.com.cn>
|
2026-06-10 07:59:24 +00:00 |
|
|
|
6deb05e0e4
|
[Core][Model] Gemma4: Unified FA4 for all layers + FlashAttention mm_prefix support (#42175)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-06-09 17:45:39 -07:00 |
|
|
Jared Wenandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
32f34d3935
|
[feature] add index share feature for DSA MTP (#44420)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-06 22:04:14 -07:00 |
|
|
|
7fe7800fa4
|
[BUG] Fix FP64 Gumbel precision coverage (#43150)
Signed-off-by: tianyu-z <zhangtianyupro@gmail.com>
Signed-off-by: Tianyu Zhang <53099276+tianyu-z@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-06-05 19:04:14 +08:00 |
|
|
DimaandDmytro Kuntso
|
128adabfe0
|
[Bugfix] Fix Gemma4 MTP block_table batch_size mismatch under concurrent load (#43982)
Signed-off-by: Dmytro Kuntso <dkuntso@amazon.co.uk>
Co-authored-by: Dmytro Kuntso <dkuntso@amazon.co.uk>
|
2026-06-03 17:11:10 -07:00 |
|
|
Luciano MartinsandLuciano Martins
|
a248b45d05
|
[Model] Add Gemma4 Unified (encoder-free) support (#44429)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
|
2026-06-03 12:01:39 -07:00 |
|
|
|
b690b2bb67
|
[Model]Support Step-3.7-Flash (#43859)
Signed-off-by: luotingdan <luotingdan@stepfun.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: luotingdan <luotingdan@stepfun.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Yu Huang <yuhuang@nvidia.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-05-28 17:01:48 -07:00 |
|
|
Benjamin Chislett
|
9202ea6fda
|
[Spec Decode] Allow causal DFlash (#43445)
|
2026-05-28 21:18:44 +00:00 |
|
|
TJian
|
1806d1adfc
|
[ROCm] [DSv4] [Perf] Support DeepSeek v4 MTP (#43385)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-05-24 18:43:08 +08:00 |
|
|
Benjamin Chislettandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
4e2eba28be
|
[Perf] Optimize hidden state extraction logic (#37374)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-22 18:23:08 -04:00 |
|
|
Benjamin Chislett
|
0fe7550254
|
[Bugfix] DFlash FP8 KV-Cache (#42692)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-05-15 08:29:45 -06:00 |
|
|
Isotr0pyandzxy
|
faa4b76afa
|
[Model] Support InternS2 Preview (#42705)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: zxy <46674730+CUHKSZzxy@users.noreply.github.com>
|
2026-05-14 21:30:26 -07:00 |
|
|
Siddharth Bedekar
|
f51f6844f9
|
[Bugfix][Spec Decode] Wire draft_probs into probabilistic draft_model rejection (#40269)
|
2026-05-13 21:04:03 -04:00 |
|
|
Michael GoinandClaude Opus 4.6
|
2f821faeae
|
[Spec Decode] Support hybrid attention models in extract_hidden_states (#39949)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-05-13 10:45:53 -07:00 |
|