|
c6843d397c
|
[Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771)
Signed-off-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-29 10:36:22 +08:00 |
|
|
qizixi
|
6d3ea3c2c9
|
[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-parallel push prefill (#50499)
Signed-off-by: zixi-qi <zixi@inferact.ai>
|
2026-09-28 18:50:46 +02:00 |
|
|
linitra24andJee Jee Li
|
953f90d25f
|
[LoRA] Support variable num_labels for sequence classification (#57766)
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-09-28 14:58:35 +00:00 |
|
|
Chao Lei
|
386ac2573e
|
[Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout (reopens #55763) (#58919)
|
2026-09-27 19:47:36 +00:00 |
|
|
Lio Einaudiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
0c87a197b8
|
[Bugfix] Disable sequence parallelism / async TP under batch invariance and add a TP regression test (#56377)
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-26 22:19:08 -04:00 |
|
|
Yousafandsfeng33
|
83db8c8390
|
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957)
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
|
2026-09-25 20:43:08 +00:00 |
|
|
|
73a78e6f1f
|
[SpecDecode] Add LiLiCorr drafter (#57934)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Andrii Skliar <andreyws96@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-09-25 10:08:28 -04:00 |
|
|
Isotr0py
|
24928a4d62
|
[UX][Frontend] Introduce vllm preload cli for fast restart (#56680)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-25 06:56:39 +00:00 |
|
|
|
90a9515006
|
[Cleanup] Remove online quantization support in fp8.py in favor of online shorthands (#53585)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 12:48:14 -07:00 |
|
|
Lio Einaudi
|
711fc55c10
|
[Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M (#57586)
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
|
2026-09-23 14:39:38 -04:00 |
|
|
Tianyu GuoandCyrus Leung
|
3ee2906f47
|
[EPD] Support metadata-only audio inputs (#57887)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-23 09:03:50 +08:00 |
|
|
|
639461eda4
|
[RL][Sleep] Retain frozen weights across level-2 sleep (#57891)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-22 12:36:55 +08:00 |
|
|
|
13058faa49
|
[Frontend] Add reusable TP1 initialized-engine snapshots (#51360)
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-21 23:18:49 +00:00 |
|
|
Nicolò Lucchesi
|
fba33e39cb
|
[Docs] Add an Engram feature page explaining Engram usage in vLLM (#57910)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-21 17:15:28 +00:00 |
|
|
jiangkuaixue123andCodex
|
7ddd461a28
|
[Docs] Add ECMooncakeConnector usage example for EPD (#57742)
Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-21 09:15:02 +08:00 |
|
|
roikoren755andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
49ee12d742
|
[Misc] Rename --enable-mamba-fine-grained-prefix-cache (#53945 follow-up) (#57382)
Signed-off-by: Roi Koren <roik@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-20 09:50:34 -07:00 |
|
|
|
5302d1fe40
|
[Model Runner V2] Support custom logits processors (#56497)
Signed-off-by: David Cheung <d7cheung@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-09-19 17:46:49 +00:00 |
|
|
Nicolò Lucchesi
|
164dca8d1e
|
[Nixl] Separate transport-failure metrics from KV expiry (#55854)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-18 18:08:59 +02:00 |
|
|
|
dee37d8911
|
[Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 04:32:39 +00:00 |
|
|
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
|
YoungandYoung
|
b78fbab4be
|
[Docs] Fix the typos in the document (#57190)
Co-authored-by: Young <yangchh15@163.com>
|
2026-09-17 15:52:10 +00:00 |
|
|
|
2743dc4f92
|
[Feature][Multimodal] Add cross-encoder caching to Mooncake P2P (#56242)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Co-authored-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-17 13:16:41 +08:00 |
|
|
xinnywinneandBenjamin Chislett
|
fbf2c5e8be
|
[Frontend] Add per-request metrics to Responses API (#55084)
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-09-16 22:52:05 +00:00 |
|
|
linitra24andJee Jee Li
|
c81eb3e69c
|
[LoRA] Support modules_to_save for sequence classification (#53555)
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-09-16 07:15:42 -07:00 |
|
|
alberto
|
75dc588269
|
[KV Offload] Add per-request max_load_tokens control (#55885)
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
|
2026-09-16 06:57:38 +03:00 |
|
|
|
0fefffc934
|
[KVConnector][NIXL] Support attention-HMA layouts in pipeline-parallel push prefill (#50494)
Signed-off-by: zixi-qi <zixi@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-15 15:43:56 -07:00 |
|
|
liranschour
|
17bc3e106a
|
[KVConnector][P2P] Configurable unbound-store timeout and one-RTT rejection of a late fetch (#53453)
Signed-off-by: Liran Schour <lirans@il.ibm.com>
|
2026-09-15 05:38:18 +03:00 |
|
|
Netanel Haber
|
d4ee7fe7a9
|
[Quantization] Select linear backends per quantization (#51204)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
|
2026-09-14 16:24:18 -04:00 |
|
|
Nicolò LucchesiandCodex
|
1713a09491
|
[Rust Frontend] Forward per-request watermarking controls (#56338)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-14 23:17:02 +08:00 |
|
|
|
a5f6f61a8a
|
[CI][CPU] Add speculative-decoding coverage to CPU CI (#54934)
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
|
2026-09-14 19:51:38 +08:00 |
|
|
|
7ee8a6dd01
|
[watermarking] Dual-key gumbel-max watermarking for speculative decoding support (#56122)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Signed-off-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Tomas Ruiz <tomas.ruiz.te@gmail.com>
|
2026-09-12 18:42:01 +00:00 |
|
|
Walter Beller-MoralesandFlora Feng
|
756794a9a7
|
[Frontend] create unified Cohere parser (#56392)
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-09-12 04:04:46 -04:00 |
|
|
djramic
|
5392fbca2a
|
[ROCm][Docker] Pin AINIC apt repo to snapshot 1.117.5-a-77 (#56459)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
|
2026-09-11 14:14:16 +00:00 |
|
|
|
e83d57d8ae
|
[Doc] Sync KV event medium terminology after #48123 (#51646)
Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-09-10 14:42:31 +03:00 |
|
|
|
ea40bb9e90
|
[feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-09 21:00:15 -07:00 |
|
|
Ronen Schaffer
|
138d137b5b
|
[Refactor][kv_offload]: rename block→chunk (#52615)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
|
2026-09-09 17:45:49 +03:00 |
|
|
omerpaz95
|
a97dacb710
|
[Docs][EC Connector] CPU EC Connector usage Docs (#54522)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
|
2026-09-08 17:43:08 -07:00 |
|
|
shikamd123
|
1a522b6949
|
[ROCm] [Docker] Upgrade default AINIC repo to ship libionic 54.0-187-1 (#54112)
Signed-off-by: Shiksha Patel <shiksha.patel@amd.com>
|
2026-09-08 19:21:18 +00:00 |
|
|
|
263c4ff95f
|
[Bugfix][Spec Decode] Cache the Mamba state at the block-grid position of EAGLE resume (#53945)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Adam Shaver <ashaver@nvidia.com>
Signed-off-by: akshaver <168006157+akshaver@users.noreply.github.com>
Co-authored-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: roikoren755 <26850796+roikoren755@users.noreply.github.com>
|
2026-09-08 09:59:58 -04:00 |
|
|
Sean Westfall
|
58ad1f3b89
|
[Docs] Add OLMo 2 to batch-invariance tested models (#55691)
Signed-off-by: Sean Westfall <sean.westfall@gmail.com>
|
2026-09-07 09:56:49 +00:00 |
|
|
Andreas Karatzas
|
f7f060d253
|
[Bugfix][Audio] Restore soundfile-first automatic decoding (#55642)
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-09-07 06:21:02 +00:00 |
|
|
|
5690b02c03
|
[Quantization] Support online quantization with partially pre-quantized checkpoints (#51392)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-09-04 11:46:06 -07:00 |
|
|
|
6a039f465e
|
[feat] add torchcodec as audio loader and implement selective audio backend (#51826)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: saltman155 <saltman155@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-04 16:16:10 +00:00 |
|
|
|
ffe3bb3c72
|
[Hardware][XPU] Register matmul and linear batch-invariant kernels for XPU (#49209)
Signed-off-by: tzielinski-habana <tomasz.zielinski@intel.com>
Signed-off-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-02 13:31:38 +00:00 |
|
|
fxmarty-amd
|
7894394b07
|
[Online quantization] Add targeted online quantization configuration based on user patterns (#51285)
|
2026-09-02 08:55:15 -04:00 |
|
|
周峥andGuanyi Chen
|
c866ba9d11
|
[KV Connector] Support heterogeneous TP sharing in Mooncake Store Connector (#53129)
Signed-off-by: Guanyi Chen <939416532@qq.com>
Signed-off-by: z-zanez <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: Zheng Zhou <zhouzh93@mail2.sysu.edu.cn>
Signed-off-by: zane <zhouzh93@mail2.sysu.edu.cn>
Co-authored-by: Guanyi Chen <939416532@qq.com>
|
2026-09-01 10:18:06 -07:00 |
|
|
|
40824284bc
|
[Doc] Document FP8 GEMM kernel selection and Blackwell support (#49936)
Signed-off-by: harjoth <harjoth.khara@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-09-01 13:22:40 +01:00 |
|
|
Jan Kwieciński
|
ec32f669bb
|
[Feature][MM_UUIDs] Allow empty video URLs when using multi-modal UUIDs (#54220)
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com>
|
2026-09-01 05:55:27 +00:00 |
|
|
Lain
|
882ca8d696
|
[Kernel] add Flashinfer cutedsl w4a16 linear (#53014)
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
|
2026-09-01 02:49:14 +00:00 |
|
|
macheroandClaude
|
87b9b5b8d9
|
[Test][VLM] Add batch-invariance tests for Qwen3-VL (#53531)
Signed-off-by: machero <sr1123640465@126.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-30 09:49:41 -04:00 |
|