763e3f7c91
[Bugfix][Spec Decode] Disable dynamic speculative decoding for fixed-K proposers instead of crashing ( #46399 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-10-17 12:41:04 -04:00
Stefano Castagnetta and Kevin H. Luu
784d1fff0c
[MRV2][Spec Decode] Quantized draft lm_head for drafters that share the target head ( #59973 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 22:06:40 +00:00
d92a4f25f2
[watermarking] Add DFlash support to dual-key gumbel-max watermarking ( #59554 )
...
Signed-off-by: Shern Shiou Tan <shernshiou@gmail.com >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-15 22:06:06 +00:00
7cd7982dfd
[Spec Decode] Don't reserve DFlash/DSpark draft slots in the scheduler token budget ( #59468 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-16 01:23:58 +08:00
Timofey and Codex
716c5225ae
[Model] Add AliceAI support ( #57963 )
...
Signed-off-by: timka <timka.byzov@yandex.ru >
Co-authored-by: Codex <noreply@openai.com >
2026-10-15 10:15:03 +00:00
Misha Goin and Claude Opus 5.5
1c4c319792
[Frontend] Advertise --enable-x/--disable-x for boolean CLI flags ( #61479 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 01:08:02 -07:00
48f17b77bb
[Bugfix] Reject mismatched float16/bfloat16 KV cache dtype ( #60885 )
...
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 08:06:59 +00:00
Shanshan Shen
f46021a7fa
[ROCm][Perf] Enable DPA+ETP prefill compute/comm dual-batch-overlap for DeepSeek-V4 ( #57773 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-10-15 11:16:48 +08:00
Woosuk Kwon and Claude Opus 5.5
8530aed2b7
[Spec Decode] Remove Medusa and MLP speculator support ( #61701 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 17:29:23 -07:00
Luca Motz and Claude Opus 5.5
2d09fdcdc2
[Quantization] Auto-disable DeepGEMM for Kolibri 1 on SM12x ( #61274 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 12:59:50 +00:00
3ccc0a90b6
[KV Connector] Support sleep mode with MooncakeConnector over RDMA ( #59625 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Dakai An <dakaian108@gmail.com >
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com >
2026-10-14 04:18:47 +00:00
2cc2e68d1c
[Core][Mamba] Remove leftovers of mamba_cache_mode="all" ( #60043 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Jiangyun Zhu <zhujiangyun@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-13 00:58:59 -07:00
Maroon Ayoub and Claude
d6aca837c3
[Attention] Make the sparse MLA prefill workspace size configurable ( #61023 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-12 14:56:51 -04:00
Woosuk Kwon and Claude Opus 5.5
e1a5086256
[Attention] Remove cascade attention ( #61126 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 10:41:07 -07:00
Nick Hill
7d0b4e57aa
[ROCm] Remove MRV1 model exceptions ( #56360 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-10-10 05:09:19 +00:00
a6f43e57d6
[ROCm] Enable the cuMem CUDA-graph pool offload on ROCm ( #59523 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: indianspeedster <cspandey016@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: indianspeedster <cspandey016@gmail.com >
2026-10-09 08:29:55 +00:00
e41d5d5543
[Bugfix] Don't warn about missing watermark config for default requests ( #60412 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-08 15:29:20 +00:00
Woosuk Kwon
194bd72be7
[Compile] Remove torch.compile-based sequence parallelism and AsyncTP ( #60517 )
2026-10-08 08:28:23 -07:00
819852df6b
[ModelRunner V2] Speculative Decoding NGram GPU Implementations ( #40704 )
...
Signed-off-by: PatchouliTaisa <patchychen@tencent.com >
Signed-off-by: PatchouliTaisa <pyramkar@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: PatchouliTaisa <patchychen@tencent.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
2026-10-06 13:32:59 -07:00
Nishant Revur and Claude
ad0f67a7ec
[Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 ( #59541 )
...
Signed-off-by: Nishant Revur <kumrfni@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 18:03:44 +00:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
877ddcc456
[Bugfix][Spec Decode] Reserve the bonus KV slot for fill-in DSpark ( #59105 )
...
Signed-off-by: Evgeny Savinov <notime.sea@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com >
2026-10-05 17:35:51 +00:00
Matthew Bonanni and Claude Opus 5.5
54d93af9fb
[Bugfix][HiSparse] Reject cudagraph_mode=FULL at startup ( #59688 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 12:42:27 -04:00
aoshen02 and Claude Opus 5.5
5e56e9af2a
[Feature] Release the CUDA graph pool on sleep ( #59160 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-04 01:13:10 +00:00
cb6458c5ce
[Core] Remove deprecated mamba_cache_mode "all" ( #58997 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-10-01 02:59:37 -07:00
70be6aae19
[Feature][Spec Decode] Support sampling mask replay for MRV2 MTP ( #59359 )
...
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 13:51:39 +08:00
Andrii Skliar and Andrii Skliar
deb148e730
[Mamba] Add FlashInfer ReplaySSM support for MTP ( #52928 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-09-30 21:41:17 +00:00
0f8b398158
[watermarking] add context deduplication support to speculative decoding ( #56807 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-30 12:22:10 -04:00
90e13fc757
[Bugfix][HiSparse] Never allocate GPU pages without host backing ( #59036 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 04:13:29 +00:00
Mia Ojeda
e006d761a5
[Bugfix][CPU][DiffusionGemma] Support narrower canvas w/ sync scheduling ( #59107 )
...
Signed-off-by: ojeda-e <estefania.b.ojeda@gmail.com >
2026-09-29 23:28:50 -04:00
jack and QwertyJack
79a0307d74
[Config] Remove Engram CUDA-alike device restrictions ( #59171 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
2026-09-29 12:42:02 +00:00
4e59d53ab5
[MRV2] Support stock torch.compile mode ( #59079 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-28 20:17:48 -07:00
Almog Tavor
171d1b9ef6
[Elastic EP] Support Model Runner V2 ( #53934 )
...
Signed-off-by: almogtavor <almogtavor@gmail.com >
Signed-off-by: Almog Tavor <70065337+almogtavor@users.noreply.github.com >
2026-09-29 00:20:40 +00:00
Xiaoshuang Wang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ef42093a36
[Model Runner V2][Spec Decode] Support spec decode with draft model ( #43091 )
...
Signed-off-by: Icey <1790571317@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 19:38:05 +00:00
Lio Einaudi and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
0c87a197b8
[Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test ( #56377 )
...
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-26 22:19:08 -04:00
Linkun
614b55f0b6
[AuxOutput] Only require Model Runner V2 on GPU platform ( #58205 )
...
Signed-off-by: Linkun Chen <github@lkchen.net >
2026-09-25 15:55:21 -07:00
73a78e6f1f
[SpecDecode] Add LiLiCorr drafter ( #57934 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-09-25 10:08:28 -04:00
Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
stefankoncarevic and Andreas Karatzas
e33de821c0
[ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the config ( #58535 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-24 15:36:36 -05:00
Nick Hill and Kimi
074d57fce1
[Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on ( #58593 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-24 12:07:32 -07:00
Harry Mellor and Claude Opus 5.5
d881812500
[Chore] Use Transformers v5 names and drop redundant processor use_fast ( #58550 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 14:14:10 +00:00
Lio Einaudi
711fc55c10
[Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M ( #57586 )
...
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com >
2026-09-23 14:39:38 -04:00
b715699df4
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE ( #50212 )
...
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com >
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 11:11:57 -05:00
Tianyu Guo
c961121519
[Bugfix] Disable prefix caching for encoder-only before model config hooks ( #58287 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-23 09:25:32 +00:00
826e300cde
[Bugfix] Skip VllmConfig re-validation for with_hf_config submodel views ( #58212 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Roger Wang <rogerw@inferact.ai >
2026-09-23 00:50:17 +00:00
Misha Goin
5c3b61e9c4
[Core] Disable JIT warmup in eager mode ( #58197 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-22 17:35:23 -04:00
1b3b88ec2b
[Core] structured generation mode for DiffusionGemma model (Jev-like) ( #57250 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Razorback16 <razorback16@protonmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Razorback16 <razorback16@protonmail.com >
2026-09-22 20:51:17 +00:00
aoshen02
feb87b1934
[Bugfix] Narrow AuxOutput KV restrictions to known PD connectors ( #58150 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-09-22 23:09:39 +08:00
Artem Perevedentsev and CNE Pierre FICHEPOIL
1ea7c63f4a
[Bugfix][Structured Output] Disallow MRV1 + PP>1 + async sched + structured output ( #56250 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: CNE Pierre FICHEPOIL <pierre-1.fichepoil@gendarmerie.interieur.gouv.fr >
2026-09-22 18:00:05 +04:00
Jeff (Junze) Ma
0961bbae28
[Spec Decode] Enable async scheduling for DFlash ( #58065 )
2026-09-21 22:05:44 -07:00