David Zheng
1acd67a795
[Bugfix] Fix XPU/ROCm compatibility in spawn_new_process_for_each_test ( #41895 )
...
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com >
2026-05-08 00:47:22 -04:00
Hongxia Yang and TJian
20cac26b19
[ROCm] Enable SimpleCPUOffloadConnector on ROCm ( #40549 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-05-06 20:52:02 -07:00
27702f6d08
[Bugfix] Fix token loss in PP mode which causes degraded accuracy ( #41133 )
...
Signed-off-by: Jing Wang <jingwang96@qq.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-06 14:07:32 -04:00
David Zheng
ee38750a75
[Bugfix] Fix spawn_new_process_for_each_test silently swallowing test failures ( #41423 )
...
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com >
2026-05-06 11:17:15 -04:00
Luciano Martins and Luciano Martins
27e0057aed
[Spec Decode] Add Gemma4 MTP speculative decoding support ( #41745 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
2026-05-06 22:39:29 +08:00
Ronen Schaffer
f39bcf1e30
[KV Offload] Return None from lookup() for in-flight blocks ( #41795 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
2026-05-06 17:31:21 +03:00
zhanqiuhu and NickLucche
df8e63f4ed
nixl refactor: new transfer design ( #40731 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
Signed-off-by: NickLucche <nlucches@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
2026-05-06 06:16:25 -07:00
wang.yuqi and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
51c1ee9b7c
[Examples] Resettle Disaggregated examples. ( #40759 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-06 01:20:38 -07:00
wi-adam
b53c507bc9
[Bugfix] Skip PP sampled-token receive on last rank during async scheduling ( #40749 )
...
Signed-off-by: Adam Winstanley <adam@winstanley.industries >
2026-05-06 05:31:14 +00:00
Matthew Bonanni
01b9b5af67
[Attention] Minor refactor: layer takes ownership of the MLA prefill backend ( #41744 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-05-05 23:22:41 +00:00
Lanze Liu
79246b5ea6
[Spec Decode] Fix max_model_len logging in speculative config for draft model ( #41571 )
...
Signed-off-by: Lanze Liu <lanzetech@gmail.com >
2026-05-05 21:56:06 +00:00
Zhewen Li and Zhewen Li
bee126165f
[P/D][Mooncake] Add KVConnectorStats for transfer observability ( #40414 )
...
Signed-off-by: Zhewen Li <zhewenli@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-05-05 02:17:38 -07:00
Wentao Ye
577b9623e6
[Bug] Fix status update address for non-MOE model within external dp mode ( #40839 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-05-04 16:37:16 -07:00
Andreas Karatzas
1cb0838721
[ROCm][CI] Fix MLA prefill scale for DeepSeek GSM8K ( #41569 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-04 16:32:55 -07:00
Joachim Studnia and Claude Opus 4.6
422dd02598
[bugfix] Fix prompt logprobs on request eviction during chunked prefill ( #41411 )
...
Signed-off-by: Joachim Studnia <joachim@mistral.ai >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-04 11:46:00 -07:00
Andreas Karatzas
01d4d1ad37
[ROCm][CI] Align spec decode logprob test prefill settings ( #41335 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-04 04:33:29 +00:00
66dfee7121
[Bugfix] Fix degenerate KV cache stride causing TMA cudaErrorIllegalInstruction ( #40737 )
...
Signed-off-by: David Oy <david@baseten.co >
Signed-off-by: David Oy <58150256+the-david-oy@users.noreply.github.com >
Signed-off-by: David Oy <david.oy@baseten.co >
Co-authored-by: David Oy <david@baseten.co >
Co-authored-by: Claude <claude@anthropic.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-05-03 23:52:18 +00:00
Luka Govedič
d58c42e19c
[vLLM IR] 2/N fused_add_rms_norm and maybe_inplace overload ( #36823 )
...
Signed-off-by: Luka Govedič <lgovedic@redhat.com >
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
2026-05-01 23:41:15 -04:00
f3fef12350
[Attention] Abstract the MLA prefill backends and eliminate cuDNN ( #32623 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-01 13:36:20 -04:00
Or Ozeri
2fa1f8ec00
[kv_offload+HMA][13/N]: Enable HMA support ( #41445 )
...
This is the final PR in a series to enables HMA support for the
offloading connector. The connector advertises `SupportsHMA`
and is validated with unit tests and e2e tests.
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-05-01 12:30:03 +01:00
a3ec4a35f5
[Bugfix][Metrics] Fix RayPrometheusMetric.labels() returning shared labeled child ( #40840 )
...
When vLLM runs with Ray Prometheus `vllm:request_success{finished_reason=...}`
only ever increments the repetition bucket regardless of the request's actual finish
reason; stop, length, abort, and error stay at zero. Root cause was `labels()` mutated
the wrapped Ray metric's default tags in place and returned self, so every `.labels(...)`
call on a given wrapper returned the same object.
Co-authored-by: Marwan Sarieddine <sarieddine.marwan@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com >
Signed-off-by: Seiji Eicher <seiji@anyscale.com >
2026-05-01 08:43:39 +01:00
Bugen Zhao and wang.yuqi
a07642667d
[Bugfix] Pass reasoning parser kwargs to structured output ( #41199 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-30 23:38:02 -07:00
sychen52
947138b6c2
Add nvfp4 kv cache support ( #40177 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
2026-05-01 04:55:16 +00:00
Luis 🚀 and Andrew Sansom
14043dfecd
feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API ( #40720 )
...
Signed-off-by: Luis Robaina <luis@protopia.ai >
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com >
Signed-off-by: LuisRobaina <luis@protopia.ai >
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com >
2026-05-01 10:05:55 +08:00
Nick Hill
dd5506a157
[Core] Simplify handling of scheduler_reserve_full_isl option ( #41064 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-30 18:10:00 -07:00
Stefano Castagnetta and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
92a7c121b6
[CI] Add MTP coverage: Qwen3.5 correctness + no-sync spec decode ( #40472 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-30 12:24:09 -07:00
snadampal
3179e53135
[P/D] Prefill compute optimizations with bi-directional KV cache transfers between P and D nodes ( #32553 )
...
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com >
2026-04-30 10:14:20 +00:00
Nicolò Lucchesi
efdc95674d
[KVConnector] MultiConnector SupportsHMA ( #39571 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-30 02:10:50 -07:00
Martin Hickey and Or Ozeri
c42981d034
[Refactor][kv_offload] KV Offloading maintainability improvements ( #40538 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-04-30 05:55:31 +03:00
Wei Zhao
0ff1bf9bb1
[Bugfix] Fix failure to allocate KV blocks error ( #41282 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-04-29 18:44:07 -07:00
Nick Hill
18599bfdf2
[Ci][BugFix] Fix slow DP tests due to bad teardown logic ( #41166 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-29 19:31:00 -04:00
0335316a9b
[BUG] Two phase pause to prevent deadlock ( #39366 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
Signed-off-by: Aaron Hao <ahao@anyscale.com >
Co-authored-by: Junjie Zhang <junj.jay.zhang@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-04-29 17:51:03 -04:00
Artem Perevedentsev
b92ef9ec5a
[Perf] Enable FlashInfer top-k/top-p sampler by default ( #40376 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-04-29 19:10:34 +04:00
68dd7db810
[Reasoning] Support for speculative decoding with thinking budget ( #34668 )
...
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com >
Signed-off-by: rishitdholakia13 <123388671+rishitdholakia13@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-04-29 06:14:52 +00:00
8a8c9b564e
[KV Offload] Per-job store completion for CPU offloading connector ( #39186 )
...
Signed-off-by: Itay Etelis <itay.etelis@ibm.com >
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com >
Co-authored-by: Itay Etelis <itay.etelis@ibm.com >
Co-authored-by: Or Ozeri <or@ozery.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-04-29 08:52:55 +03:00
liangel-02
7fd05e05ae
uncomment flex backend for batch invariant mode ( #40842 )
...
Signed-off-by: Angel Li <liangel@meta.com >
2026-04-28 21:05:14 -07:00
Nick Hill
e68fa1b90a
[Core] Account for num_gpu_blocks_override in max_model_len checks ( #41069 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-28 15:44:09 -07:00
yangrz and Roger Wang
de3fe8dc62
[Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA ( #40449 )
...
Signed-off-by: yangruize <yangruize7@163.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-28 11:38:43 -07:00
rasmith
358a755e43
[CI][AMD][BugFix] Update request URL in test_moriio_connector to match vllm-router compatibility changes ( #41076 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
2026-04-28 13:14:59 -05:00
wang.yuqi
a8208e6a81
[Examples] Resettle features examples. ( #40995 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-28 00:33:41 -07:00
7b1bc0a3eb
[Bugfix] Cap SWA/chunked-local runtime admission to startup pool-sizing bound ( #40946 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-04-27 04:33:13 +00:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
omerpaz95 and Or Ozeri
b39c266dae
[KV Offload] Offload all KV blocks when doing prefill in P/D ( #40346 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-04-26 15:06:01 +03:00
Andreas Karatzas
95995bbef8
[ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset ( #38503 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 05:25:20 +00:00
Andreas Karatzas
428b988c98
[ROCm][CI] Fix trust_remote_code AttributeError in EAGLE3 acceptance length test ( #40306 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-25 02:59:31 +00:00
Benjamin Chislett and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
626daa2076
[Feat] Unified Synthetic Acceptance Rate for V1 and V2 ( #40662 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-24 00:48:08 +00:00
Nick Hill
fe85a92e86
[Core] Avoid seq_lens_cpu GPU->CPU sync ( #40654 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-24 00:35:55 +00:00
Matthew Bonanni
cde8d24710
[Spec Decode] Move SpecDecodeBaseProposer out of eagle.py ( #40732 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-23 22:28:27 +00:00
Nicolò Lucchesi
8824f50f1f
[CI] Split disaggregated tests into own test-area ( #40623 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-23 23:20:12 +08:00
Fynn Schmitt-Ulms and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
29f64c5f5e
FlexAttention non-causal support ( #40394 )
...
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 13:22:57 -07:00
David Zheng