 Giancarlo Delfin
|
facd9a74a1
|
[Model Runner V2][Spec Decode] Skip DP sync for all speculator uniform decodes (#54856)
|
2026-09-03 10:32:33 +00:00 |
|
|
Zihan ZhangandClaude Opus 5
|
d61b6e1878
|
[Bugfix][Spec Decode] Take the DFlash draft's RoPE layout from its own config (#54373)
Signed-off-by: SubSir <tiancaizhangdaxian@sjtu.edu.cn>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-08-31 18:13:20 -04:00 |
|
|
Giancarlo Delfin
|
fe755c8899
|
[Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's (#54282)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-29 09:58:29 -07:00 |
|
|
Summer Yang
|
6d4562c59b
|
[Attention][DCP] Enable FlashInfer MLA for DSpark drafting (#54277)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
|
2026-08-28 21:11:28 -07:00 |
|
|
Giancarlo Delfin
|
5acc1c4e4b
|
[Model Runner V2][Spec Decode] Skip DP sync before EAGLE/MTP draft prefill (#53694)
|
2026-08-26 20:59:35 -07:00 |
|
|
Misha Goin
|
f620499ee3
|
[Bugfix][Spec Decode] Reapply group geometry for FlashAttention metadata (#53336)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-24 08:08:43 -07:00 |
|
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
|
Misha Goin
|
9ff7041b55
|
[Bugfix][Spec Decode] Use group geometry for FlashAttention metadata (#53002)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-21 17:07:19 -07:00 |
|
|
Wei Zhao
|
d1e3eee6fb
|
[Spec decode] Support Kimi-K3 DCP with DSpark (#52188)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-08-17 13:08:45 -07:00 |
|
|
Gabriel Wu
|
97388c44f9
|
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538)
|
2026-08-15 07:01:32 -07:00 |
|
|
|
6adad08767
|
Add Muse Glimmer model support (#51655)
Signed-off-by: Tiezhen Wang <tiezhen@inferact.ai>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Beto de Paola <betodepaola@meta.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: zixi-qi <zixi@inferact.ai>
|
2026-08-14 12:28:56 +08:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
Giancarlo Delfin
|
dec13a33b7
|
[Model Runner V2][Spec Decode] Add multi-layer MTP speculator (#48892)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-07-30 15:54:15 -07:00 |
|
|
+13
|
aeeb36b1f1
|
[New model] Kimi K3 (#50000)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-07-30 03:49:58 -07:00 |
|
|
Fynn Schmitt-UlmsandMichael Goin
|
642076d26c
|
Support loading sample_from_anchor flag from speculators config (#48639)
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
|
2026-07-20 14:52:43 -07:00 |
|
|
|
4ec199b66a
|
[Bugfix][Spec-Decode] Populate draft seq_lens_cpu_upper_bound for spec-decode attention metadata (#44492)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Signed-off-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: okorzh-amd <okorzh-amd@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-20 20:50:58 +00:00 |
|
|
Michael GoinandClaude Opus 4.8
|
8bfd683901
|
[Spec Decode] Add kv_cache_dtype to speculative_config to control separately from target (#48787)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-15 23:49:23 -06:00 |
|
|
Michael GoinandClaude Fable 5
|
ecf4aa5ce2
|
[Bugfix] Fix FlashInfer non-causal draft attention (DFlash/DSpark) on Blackwell (#48167)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-15 12:44:01 -07:00 |
|
|
Nick Hill
|
26587f9519
|
[BugFix][ModelRunner V2] Fix stale attn metadata in speculator prefill cudagraph capture (#48261)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-13 09:39:15 -07:00 |
|
|
Michael GoinandClaude Opus 4.8
|
0d12618e98
|
[Spec Decode] Support hybrid (SWA + full attention) DFlash drafters (#47914)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-08 11:12:45 -04:00 |
|
|
Benjamin Chislett
|
7bd154375d
|
[Bugfix] Fix mamba+dflash for MRV2 (#47698)
|
2026-07-07 15:59:13 -04:00 |
|
|
|
f5a8d73377
|
[Spec Decode] DSpark (#46995)
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-01 08:30:24 -07:00 |
|
|
Giancarlo Delfin
|
3cecee40f3
|
[Model Runner V2][Spec Decode] Fix stale values in idx_mapping from CG num reqs padding (#47066)
|
2026-06-30 11:25:32 -07:00 |
|
|
Ilya MarkovandMarkov Ilya
|
a2abce646f
|
[EPLB] Mask padding in EPLB load recording (#38128)
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
|
2026-06-28 19:43:58 -07:00 |
|
|
Giancarlo Delfin
|
5314665bad
|
[Model Runner V2][DFlash] Enable dflash attention backend selection (#46770)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-06-25 19:29:25 -07:00 |
|
|
Giancarlo Delfin
|
ef361de916
|
[Model Runer V2][DFlash] Fix lm head sharing for dflash (#46435)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-06-23 19:09:06 +00:00 |
|
|
Benjamin ChislettandGiancarlo Delfin
|
0bae1d3848
|
[MRV2][Spec Decode] DFlash (#44586)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-06-10 08:47:46 -07:00 |
|