|
73a78e6f1f
|
[SpecDecode] Add LiLiCorr drafter (#57934)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-09-25 10:08:28 -04:00 |
|
|
|
90a9515006
|
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 12:48:14 -07:00 |
|
|
Nick Hill
|
09fe178dba
|
Revert "[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)" (#58484)
|
2026-09-24 08:48:36 -07:00 |
|
|
Wentao Ye
|
3c42385d7d
|
[Refactor] Remove dead or duplicate tests (#58446)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-09-24 10:29:56 -04:00 |
|
|
Harry MellorandClaude Opus 5.5
|
d881812500
|
[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-09-24 14:14:10 +00:00 |
|
|
|
157bcb7c48
|
[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-09-23 04:52:28 -07:00 |
|
|
Nick Hill
|
afec265bca
|
[MRV2] Miscellaneous code cleanup (#57980)
|
2026-09-23 01:51:52 -07:00 |
|
|
Michael Lapshin
|
d421a2e8fc
|
[Perf] Remove CPU-GPU sync in heterogeneous vocabulary speculative decoding (#57396)
|
2026-09-22 18:34:19 -04:00 |
|
|
lucamotz
|
276db94fb2
|
[Bugfix][Model][Spec Decode] Defer disposable GLM MTP head (#55442)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
|
2026-09-22 20:08:27 +00:00 |
|
|
Yicong Wang
|
c4d424c2e2
|
[Bugfix][Spec Decode] Cap DFlash/DSpark profiling query batch (#56448)
Signed-off-by: wangyicong <wangyicong@bytedance.com>
|
2026-09-22 12:36:32 +08:00 |
|
|
siyuandIsotr0py
|
c723a831a8
|
[Fast Start] Cache the MTP draft model in a separate daemon group (#57312)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-22 03:13:38 +00:00 |
|
|
Misha Goin
|
db7a24c230
|
[Perf] Add fused DFlash2 grouped convolution (#55960)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-09-17 21:38:59 +00:00 |
|
|
Doug Brown
|
2b02c6c29b
|
[Model] Add Cohere2MoE Eagle3 auxiliary hidden states (#49819)
Signed-off-by: Doug Brown <doug@github.douggo.com>
|
2026-09-17 20:09:36 +00:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
yzong-rh
|
e6960af33b
|
[Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200)
|
2026-09-15 09:48:49 -04:00 |
|
|
Giancarlo Delfin
|
3bb0a03f35
|
[Model Runner V2] Acceptance estimation for adaptive verification (#52228)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-14 15:58:48 -07:00 |
|
|
|
82a85dc1d2
|
[Attention] Add Triton/FlashInfer composite for multimodal prefix attention (#56305)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 14:29:41 +00:00 |
|
|
Woosuk KwonandOpenAI Codex
|
2671fedfc7
|
[MRV2] Revert explicit Triton JIT warmup migration (#56654)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-13 04:07:56 +00:00 |
|
|
|
aed894c190
|
[6/N][warmup][DSv4] Migrate sampling, and DFlash JIT kernels (#56323)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-12 17:24:24 -07:00 |
|
|
|
7ee8a6dd01
|
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-09-12 18:42:01 +00:00 |
|
|
Canlin GuoandCodex
|
dca96bf97b
|
[Bugfix][Spec Decode] Avoid fastsafetensors deadlock for PP draft models (#54416)
Signed-off-by: Canlin <canlinguosdu@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-12 14:22:30 +08:00 |
|
|
Giancarlo Delfin
|
9dcf6bf344
|
[BugFix] Fix DP token padding in dflash attention metadata (#56181)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-09-11 14:33:53 -07:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
9dd969da09
|
[Model][ROCm] Enable DeepSeek V4 Vision (#55107)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-11 09:17:30 -07:00 |
|
|
|
d0dfe587d5
|
[Bugfix] Fall back to full decode graphs for noncompiled models (#55095)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-09-11 01:56:05 -07:00 |
|
|
|
980c16c8e4
|
[PCP][Spec Decode] Adds PCP support for single-module MTP and replicated DSpark. (#56107)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-10 22:07:53 -07:00 |
|
|
|
83990f5bcc
|
[Bugfix][MRV2] Initialize DCP metadata after batch partitioning (#55212)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Ziyang Zhang <hafuhafu@qq.com>
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: Ziyang Zhang <hafuhafu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-10 06:50:09 +00:00 |
|
|
|
83fe99399e
|
[Spec Decode] Add NVFP4 DSpark gathered top-k projection (#55713)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-09 17:33:46 +00:00 |
|
|
|
9c2d21046b
|
[Bugfix][Spec Decode] Honour the draft's moe_backend on Model Runner V2 (#54788)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-08 16:50:07 -07:00 |
|
|
Flora FengandWentao Ye
|
29f46ac475
|
[Perf] Eliminate full-history reasoning scans for structured outputs (#55223)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-09-08 14:07:16 -07:00 |
|
|
Stefano Castagnetta
|
2524051385
|
[Bugfix][Spec Decode] Honour the draft's attention_backend on Model Runner V2 (#54826)
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
|
2026-09-04 12:23:19 -07:00 |
|
|
Giancarlo Delfin
|
fe755c8899
|
[Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's (#54282)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-29 09:58:29 -07:00 |
|
|
|
d3d79ffc1e
|
[Bugfix][Spec Decode] Capture the widest uniform decode batch by default (#50488)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Rahul Chalamala <rahul@modal.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <devin@cognition.ai>
|
2026-08-28 22:38:13 -07:00 |
|
|
Summer Yang
|
6d4562c59b
|
[Attention][DCP] Enable FlashInfer MLA for DSpark drafting (#54277)
Signed-off-by: Summer Yang <girasoleyang@gmail.com>
|
2026-08-28 21:11:28 -07:00 |
|
|
Qiming Zhang
|
c01b50e390
|
[Test] Fix mock_current_vllm_config missing kernel_config in test_dflash2 (#54132)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
|
2026-08-27 22:28:43 -07:00 |
|
|
Misha Goin
|
7de96050c5
|
[Spec Decode] Enable adaptive DSpark on SM100 sparse MLA (#52783)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-08-25 13:31:15 -04:00 |
|
|
Karan Shelar
|
a9a17e7095
|
Dflash2 load fix (#53435)
|
2026-08-24 21:32:43 -07:00 |
|
|
Zihan Zhang
|
b389ac2946
|
[Spec Decode] DFlash2: local convolution + candidate selector (#52816)
|
2026-08-20 22:27:22 -07:00 |
|
|
Chauncey
|
c6e19b3be2
|
[Bugfix][Structured Output] Avoid spurious FSM errors after speculative reasoning end (#53046)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-08-21 10:01:06 +08:00 |
|
|
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
|
Flora Feng
|
12f64b39d2
|
[Bugfix][Structured Output] Stop XGrammar token batches at termination (#52805)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-08-18 21:21:43 +00:00 |
|
|
Wei Zhao
|
d1e3eee6fb
|
[Spec decode] Support Kimi-K3 DCP with DSpark (#52188)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
|
2026-08-17 13:08:45 -07:00 |
|
|
|
4d2a68d64d
|
[Bugfix][Spec Decode][Structured Output] DSpark: fix the grammar bitmask mapping when the draft budget is zero (#52436)
Signed-off-by: oops-oom <73481342@qq.com>
Co-authored-by: oops-oom <73481342@qq.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-08-16 11:09:23 +00:00 |
|
|
Gabriel Wu
|
97388c44f9
|
[Bugfix] Make DSV4 sparse MLA work end-to-end for plain decode, MTP, and DSpark (#51538)
|
2026-08-15 07:01:32 -07:00 |
|
|
|
7f7a32cfec
|
[Spec Decode] DSpark confidence-scheduled verification (#47808)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Signed-off-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-12 12:27:48 -07:00 |
|
|
Lucas WilkinsonandOpenAI Codex
|
02ac17851d
|
[Refactor][MRV2] Unify uniform decode token count helper (#51917)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-12 05:58:43 +00:00 |
|
|
|
0a94d85a66
|
[Bugfix][MRV2] Require all requests to be decoding for uniform-decode dispatch (#51865)
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com>
Co-authored-by: Janelle Cai <janelle.cai@modal.com>
|
2026-08-11 17:15:06 -07:00 |
|
|
Giancarlo DelfinandNick Hill
|
27930df9c2
|
[Model Runner V2] Fix -1 placeholder draft token ids in rejection sam… (#50939)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-06 16:32:00 -07:00 |
|
|
Giancarlo Delfin
|
81bc196913
|
[Model Runner V2] Cache draft logits in model's LM head dtype (#50910)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-05 19:12:46 -07:00 |
|
|
gabriel-peracioandKimi K3
|
47a4e410ba
|
[Bugfix][Spec Decode] Fix NaN handling in rejection sampler tl.argmax (#50183)
Signed-off-by: Gabriel Peracio <gabriel@sigsonic.com>
Co-authored-by: Kimi K3 <noreply@moonshot.ai>
|
2026-08-05 18:01:09 -07:00 |
|