Yongye Zhu
edd60ac93a
[Bugfix] Fix persistent_topk inter-CTA init race on RadixRowState ( #41444 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-05-01 14:42:52 -07:00
Yongye Zhu
bcf5cac9fb
[DSV4] Add knob to enable pre-attn gemm ( #41443 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
2026-05-01 12:23:17 -07:00
Isotr0py and Roger Wang
a9484dac7b
[Perf] Intergrate Tile Kernels head_compute_mix_kernel for Deepseek-V4 ( #41255 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-05-01 15:01:17 -04:00
f3fef12350
[Attention] Abstract the MLA prefill backends and eliminate cuDNN ( #32623 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-01 13:36:20 -04:00
Wentao Ye and Nick Hill
51295793a2
[Model Runner V2] Add logprob_token_ids support ( #40559 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-05-01 10:02:03 -07:00
Michael Goin
3ccc1ff495
[Eval][CI] Add basic mrcr eval to tests/evals/ ( #40164 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-05-01 12:00:38 -04:00
529c671e80
[ROCm][FEAT] AITER Fused Allreduce + RMSNorm ( #37646 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com >
Signed-off-by: junkang1991 <junkangchow@gmail.com >
Co-authored-by: Rita Brugarolas <Rita.BrugarolasBrufau@amd.com >
Co-authored-by: junkang1991 <junkangchow@gmail.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-05-01 23:07:18 +08:00
Pleaplusone and Matthew Wong
bc635fad23
[ROCm][Deepseek] dsv3.2 further optimization ( #41217 )
...
Signed-off-by: ganyi <ygan@amd.com >
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com >
2026-05-01 23:06:00 +09:00
Artem Perevedentsev and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
c3e64696cd
[Perf] Warmup forward_native sampler kernel ( #41375 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-01 18:04:11 +04:00
sungsoo ha
4f7bde572a
[Kernel] Pack output and LSE in DCP A2A ( #41160 )
2026-05-01 09:01:17 -04:00
Or Ozeri
2fa1f8ec00
[kv_offload+HMA][13/N]: Enable HMA support ( #41445 )
...
This is the final PR in a series to enables HMA support for the
offloading connector. The connector advertises `SupportsHMA`
and is validated with unit tests and e2e tests.
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-05-01 12:30:03 +01:00
raviguptaamd
7075df79b3
[ROCm] Enable DBO (Dynamic Batch Optimization) on ROCm ( #34726 )
...
Signed-off-by: raviguptaamd <ravi.gupta@amd.com >
2026-05-01 09:18:30 +00:00
Yuyi Ao
0dbaf9daad
Refractor longcat loading to use AutoWeightsLoader ( #41448 )
...
Signed-off-by: George-ao <yuyiao772@gmail.com >
2026-05-01 09:07:23 +00:00
a3ec4a35f5
[Bugfix][Metrics] Fix RayPrometheusMetric.labels() returning shared labeled child ( #40840 )
...
When vLLM runs with Ray Prometheus `vllm:request_success{finished_reason=...}`
only ever increments the repetition bucket regardless of the request's actual finish
reason; stop, length, abort, and error stay at zero. Root cause was `labels()` mutated
the wrapped Ray metric's default tags in place and returned self, so every `.labels(...)`
call on a given wrapper returned the same object.
Co-authored-by: Marwan Sarieddine <sarieddine.marwan@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Signed-off-by: Marwan Sarieddine <sarieddine.marwan@gmail.com >
Signed-off-by: Seiji Eicher <seiji@anyscale.com >
2026-05-01 08:43:39 +01:00
Andreas Karatzas
32964e7700
[ROCm][CI] Upgraded UCX and RIXL ( #41210 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-01 16:40:47 +09:00
Bugen Zhao and wang.yuqi
a07642667d
[Bugfix] Pass reasoning parser kwargs to structured output ( #41199 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-30 23:38:02 -07:00
baonudesifeizhai
c3868bbbe4
[compile] Add FlashInfer FP8 async TP fusion and preserve allreduce fusion ordering #27893 ( #39505 )
...
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
2026-05-01 05:08:34 +00:00
sychen52
947138b6c2
Add nvfp4 kv cache support ( #40177 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
2026-05-01 04:55:16 +00:00
Or Ozeri
941fb50835
[kv_offload+HMA][12/N]: Scheduler-side support for sliding window groups ( #41228 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-05-01 06:59:17 +03:00
Juhi Mittal and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
6b6ac6c3c7
[Kernel][MoE] Support GELU on TRT-LLM NvFP4 fused MoE for Gemma4 ( #41050 )
...
Signed-off-by: Juhi Mittal <juhim@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-01 03:37:43 +00:00
Stefano Castagnetta
b542bdf7fb
[Bugfix] Disable FlashInfer CUTLASS MoE on SM110 (Jetson Thor AGX) ( #40808 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
2026-04-30 20:08:49 -07:00
Ronen Schaffer
415a879899
[KV Offload] Use Collection instead of Sequence/Iterable for OffloadingManager key parameters ( #41361 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
2026-05-01 05:18:38 +03:00
Dong W
7198940b39
[Model] Add Moondream3 model support(only query and caption skills) ( #32325 )
...
Signed-off-by: Dong Wang <dongw2019@gmail.com >
2026-05-01 10:06:48 +08:00
Luis 🚀 and Andrew Sansom
14043dfecd
feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API ( #40720 )
...
Signed-off-by: Luis Robaina <luis@protopia.ai >
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com >
Signed-off-by: LuisRobaina <luis@protopia.ai >
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com >
2026-05-01 10:05:55 +08:00
Andreas Karatzas
1adaa5056b
[ROCm][CI] Add ROCm score absolute tolerance floor ( #41341 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-04-30 18:59:35 -07:00
Soyaazz and Matthew Bonanni
4d5c89295b
(bugfix): block_size check for flex attn ( #41363 )
...
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-30 18:59:26 -07:00
Nick Hill
dd5506a157
[Core] Simplify handling of scheduler_reserve_full_isl option ( #41064 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-30 18:10:00 -07:00
Yongye Zhu and Roger Wang
a3c83ff2fd
Faster per-token fp8 group quant packed kernel for blackwell ( #41326 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
2026-04-30 18:09:55 -07:00
Woosuk Kwon
9c61864bf8
[DeepSeek] Use torch.mm for bf16xbf16->fp32 gemm ( #41300 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-04-30 16:28:57 -07:00
Tran Le
71725f6730
[Bugfix] Fix RoutedExpertsCapturer for Gemma 4 MoE (top_k_experts) ( #41401 )
...
Signed-off-by: Tran Le <tranle@fireworks.ai >
2026-04-30 16:19:59 -07:00
b4806c8ee1
[DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided ( #40960 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-04-30 15:33:12 -07:00
Wentao Ye
526927be94
[Model Runner v2] Fix v2 compile counter num_gpu_runner_capture_triggers and num_cudagraph_captured ( #41285 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-04-30 15:20:11 -07:00
Michael Goin
75a4c166f2
Fix typo in log message for indexer cache ( #41419 )
...
Signed-off-by: Michael Goin <mgoin64@gmail.com >
2026-04-30 15:02:14 -07:00
fxmarty-amd and Claude
2917d6363a
[NVFP4][Hopper/AMD Instinct] Add Triton kernels for NVFP4 dequantization and QDQ emulation ( #40033 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-30 17:35:48 -04:00
Stefano Castagnetta
efb4cdf2b8
[CI/Build] Skip Prithvi/Terratorch model-registry tests when terratorch is missing ( #41389 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
2026-04-30 12:47:55 -07:00
Stefano Castagnetta and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
92a7c121b6
[CI] Add MTP coverage: Qwen3.5 correctness + no-sync spec decode ( #40472 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-30 12:24:09 -07:00
Jee Jee Li
307b17ce33
[DSV4] Avoid redundant dtype conversion. ( #41374 )
...
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-30 09:57:27 -07:00
wenjun liu and Kunshang Ji
3ca6ca210f
xpu docker: pin oneAPI to 2025.3 and avoid unintended 2026 upgrade ( #41380 )
...
Signed-off-by: wendyliu235 <wenjun.liu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-30 16:02:23 +00:00
Stefano Castagnetta
10558f5f46
[CI/Build] Skip terratorch + torchgeo while PyPI has lightning quarantined ( #41377 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
2026-04-30 07:59:07 -07:00
121dbe7a22
[ROCm] ROCm DeepEP API updated to latest ( #39721 )
...
Signed-off-by: Tej Kiran <vpolamre@amd.com >
Signed-off-by: tej <37236721+itej89@users.noreply.github.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
Co-authored-by: HAIAI <39548240+HAIAI@users.noreply.github.com >
2026-04-30 07:46:59 -07:00
Matthew Bonanni
f03d82efdd
[UX][Bugfix] Fix OOM by setting PyTorch max_split_size_mb during model loading ( #41268 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-30 07:46:54 -07:00
a7fb008510
[EPLB] Optimize memory overhead in Nixl communicator ( #40013 )
...
Signed-off-by: ilmarkov <markovilya197@gmail.com >
Signed-off-by: Markov Ilya <markovilya19@gmail.com >
Co-authored-by: Markov Ilya <markovilya19@gmail.com >
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com >
2026-04-30 07:46:49 -07:00
Harry Mellor
ff449b6426
Stop mergify labelling from skipping pre-commit ( #41362 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-04-30 05:48:38 -07:00
Stefano Castagnetta and Harry Mellor
3527229517
[Doc] Fix RTD build: pytorch.org/docs/stable/objects.inv returns 404 ( #41353 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-04-30 05:06:44 -07:00
b55b26520c
[MoE] Make MoERunnerInterface a PluggableLayer for OOT support ( #35178 )
...
Signed-off-by: wxsIcey <1790571317@qq.com >
Signed-off-by: Icey <1790571317@qq.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-30 03:31:08 -07:00
snadampal
3179e53135
[P/D] Prefill compute optimizations with bi-directional KV cache transfers between P and D nodes ( #32553 )
...
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com >
2026-04-30 10:14:20 +00:00
Nicolò Lucchesi
efdc95674d
[KVConnector] MultiConnector SupportsHMA ( #39571 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-30 02:10:50 -07:00
Chenxi Qian and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
54146a9bf9
[Bugfix] correct h matrix layout in chunk_kda output kernel ( #40956 )
...
Signed-off-by: ChenxiQian <chenxi.qian.cq@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-30 16:22:41 +08:00
ca97f7b9bb
Fix Gemma4 MoE expert weight remapping ( #41206 )
...
Signed-off-by: sunghoon.baek <sunghoon.baek@connectfy.cloud >
Co-authored-by: sunghoon.baek <sunghoon.baek@connectfy.cloud >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-04-30 00:12:42 -07:00
Ekagra Ranjan
a04e0cf3b8
Fix Cohere ASR after HF upgrade ( #40582 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
2026-04-29 23:39:04 -07:00
Yongye Zhu