 Raphaël RiallandandSimon Veitner
|
c32495d58f
|
[watermark] compatibility validation (#56801)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
|
2026-10-05 14:37:13 -04:00 |
|
|
|
1388100560
|
[Kernel][LoRA] Add deterministic split-K=8 LoRA shrink for batch invariance (#59377)
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-04 18:46:16 -04:00 |
|
|
|
89439db727
|
[CI][Docs] Fix decode/prefill consistency test prefix construction; add GLM-4-9B and Phi-4 to batch-invariant tested models (#53692)
Signed-off-by: Yifan Da <1363818765@qq.com>
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Yifan Da <1363818765@qq.com>
Co-authored-by: CodeBuddy <noreply@codebuddy.dev>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-04 11:55:18 -04:00 |
|
|
aoshen02andClaude Opus 5.5
|
5e56e9af2a
|
[Feature] Release the CUDA graph pool on sleep (#59160)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-04 01:13:10 +00:00 |
|
|
Martin Hickey
|
bc21cba967
|
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-03 19:38:25 +08:00 |
|
|
Nils Matteson
|
9fdb174678
|
[Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
|
2026-10-03 03:59:17 +00:00 |
|
|
|
6e517b15c1
|
[Docs] Add ERNIE 4.5 to batch invariance tested models (#58476)
Signed-off-by: yifanFengg <evanfengff@gmail.com>
Signed-off-by: yifanFengg <yifanfen@amazon.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
|
2026-10-02 20:37:32 +00:00 |
|
|
Rajeev Warrierandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
30e956fad3
|
[Docs] Fix stale W4A16 NVFP4 default kernel in ModelOpt docs (#59135)
Signed-off-by: Rajeev Warrier <warrier.rajeev.95@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-02 13:21:43 +00:00 |
|
|
Teng MaandTeng Ma
|
1a9eaa3a68
|
[Doc] Document ECMooncakeConnector for EPD disaggregation (#59196)
Signed-off-by: Teng Ma <stmatengss@gmail.com>
Co-authored-by: Teng Ma <stmatengss@users.noreply.github.com>
|
2026-10-02 12:10:33 +08:00 |
|
|
|
9bd993564f
|
[Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-02 02:33:00 +00:00 |
|
|
Flora Feng
|
785b990116
|
[Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT (#58623)
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-10-02 02:25:48 +00:00 |
|
|
Flora Feng
|
a5105dba05
|
[Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-10-01 22:18:37 +00:00 |
|
|
Zijing Liu
|
ea84051819
|
[KVConnector][NIXL] Count completion notifications that arrive after KV expiry (#58875)
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
|
2026-10-01 21:30:58 +00:00 |
|
|
Sundri Lai
|
bcee730b1a
|
[Docs] Remove references to removed env vars (#59530)
Signed-off-by: Sundri Lai <laiyanting.neu@gmail.com>
|
2026-10-01 08:26:14 +00:00 |
|
|
Nils Matteson
|
51048b13ce
|
[Docs] Clarify snapshot runtime image support (#59127)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
|
2026-09-30 20:31:21 +00:00 |
|
|
|
0f8b398158
|
[watermarking] add context deduplication support to speculative decoding (#56807)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-09-30 12:22:10 -04:00 |
|
|
yuvalluriaandClaude Sonnet 4.6
|
863475eb9c
|
docs: add gemma-2, SmolLM2, Qwen2.5-Coder, Llama-3.2-1B to batch invariance tested models (#54441)
Signed-off-by: Yuval Luria <yluria@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-09-30 11:13:23 +00:00 |
|
|
Isotr0py
|
31876d8ea1
|
[Bugfix] Fix TorchCodec audio IO correctness (#58364)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-29 11:34:48 -05:00 |
|
|
|
dfc8e0f3e2
|
[Frontend] Support strict MiMo-V2.6 tool calling (#58019)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-29 12:47:47 +00:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
3acddf01e7
|
[Bugfix][ROCm] Keep Mooncake bootstrap ports bound during startup (#58967)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-29 04:14:26 -05:00 |
|
|
|
c6843d397c
|
[Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771)
Signed-off-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-29 10:36:22 +08:00 |
|
|
qizixi
|
6d3ea3c2c9
|
[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499)
Signed-off-by: zixi-qi <zixi@inferact.ai>
|
2026-09-28 18:50:46 +02:00 |
|
|
linitra24andJee Jee Li
|
953f90d25f
|
[LoRA] Support variable num_labels for sequence classification (#57766)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-09-28 14:58:35 +00:00 |
|
|
Chao Lei
|
386ac2573e
|
[Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919)
|
2026-09-27 19:47:36 +00:00 |
|
|
Lio Einaudiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
0c87a197b8
|
[Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test (#56377)
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-26 22:19:08 -04:00 |
|
|
Yousafandsfeng33
|
83db8c8390
|
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957)
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
|
2026-09-25 20:43:08 +00:00 |
|
|
|
73a78e6f1f
|
[SpecDecode] Add LiLiCorr drafter (#57934)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-09-25 10:08:28 -04:00 |
|
|
Isotr0py
|
24928a4d62
|
[UX][Frontend] Introduce vllm preload cli for fast restart (#56680)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-25 06:56:39 +00:00 |
|
|
|
90a9515006
|
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 12:48:14 -07:00 |
|
|
Lio Einaudi
|
711fc55c10
|
[Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M (#57586)
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
|
2026-09-23 14:39:38 -04:00 |
|
|
Tianyu GuoandCyrus Leung
|
3ee2906f47
|
[EPD] Support metadata-only audio inputs (#57887)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-23 09:03:50 +08:00 |
|
|
|
639461eda4
|
[RL][Sleep] Retain frozen weights across level-2 sleep (#57891)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-22 12:36:55 +08:00 |
|
|
|
13058faa49
|
[Frontend] Add reusable TP1 initialized-engine snapshots (#51360)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-21 23:18:49 +00:00 |
|
|
Nicolò Lucchesi
|
fba33e39cb
|
[Docs] Add an Engram feature page explaining Engram usage in vLLM (#57910)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-21 17:15:28 +00:00 |
|
|
jiangkuaixue123andCodex
|
7ddd461a28
|
[Docs] Add ECMooncakeConnector usage example for EPD (#57742)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-21 09:15:02 +08:00 |
|
|
roikoren755andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
49ee12d742
|
[Misc] Rename --enable-mamba-fine-grained-prefix-cache (#53945 follow-up) (#57382)
Signed-off-by: Roi Koren <roik@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-20 09:50:34 -07:00 |
|
|
|
5302d1fe40
|
[Model Runner V2] Support custom logits processors (#56497)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-19 17:46:49 +00:00 |
|
|
Nicolò Lucchesi
|
164dca8d1e
|
[Nixl] Separate transport-failure metrics from KV expiry (#55854)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-18 18:08:59 +02:00 |
|
|
|
dee37d8911
|
[Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 04:32:39 +00:00 |
|
|
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
|
YoungandYoung
|
b78fbab4be
|
[Docs] Fix the typos in the document (#57190)
Co-authored-by: Young <yangchh15@163.com>
|
2026-09-17 15:52:10 +00:00 |
|
|
|
2743dc4f92
|
[Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-17 13:16:41 +08:00 |
|
|
xinnywinneandBenjamin Chislett
|
fbf2c5e8be
|
[Frontend] Add per-request metrics to Responses API (#55084)
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-09-16 22:52:05 +00:00 |
|
|
linitra24andJee Jee Li
|
c81eb3e69c
|
[LoRA] Support modules_to_save for sequence classification (#53555)
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-09-16 07:15:42 -07:00 |
|
|
alberto
|
75dc588269
|
[KV Offload] Add per-request max_load_tokens control (#55885)
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
|
2026-09-16 06:57:38 +03:00 |
|
|
|
0fefffc934
|
[KVConnector][NIXL] Support attention-HMA layouts in pipeline-parallel push prefill (#50494)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-15 15:43:56 -07:00 |
|
|
liranschour
|
17bc3e106a
|
[KVConnector][P2P] Configurable unbound-store timeout and one-RTT rejection of a late fetch (#53453)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
|
2026-09-15 05:38:18 +03:00 |
|
|
Netanel Haber
|
d4ee7fe7a9
|
[Quantization] Select linear backends per quantization (#51204)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
|
2026-09-14 16:24:18 -04:00 |
|
|
Nicolò LucchesiandCodex
|
1713a09491
|
[Rust Frontend] Forward per-request watermarking controls (#56338)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-14 23:17:02 +08:00 |
|
|
|
a5f6f61a8a
|
[CI][CPU] Add speculative-decoding coverage to CPU CI (#54934)
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-14 19:51:38 +08:00 |
|