da0b2d8b17
[Performance] Optimize Dots3 NOTE runtime ( #53517 )
...
Signed-off-by: KurodaKanbei <mistergalahad@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-31 00:05:18 -07:00
Prudhvi Vuda
7292ee2791
[Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs ( #53808 )
...
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com >
2026-08-31 06:37:12 +00:00
eligotts and OpenAI Codex
4ae172231c
[Frontend] Forward cache salt for content parts ( #54315 )
...
Signed-off-by: Eli Gotts <78387377+eligotts@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-30 23:29:53 -07:00
Hotragn Pettugani
8fd9eb85d5
[Bugfix][Frontend] Truncate prompt_token_offsets with the prompt ( #54407 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
2026-08-31 06:13:37 +00:00
+5
e126687a9a
[Model] Support Qwen3.8-Flash-Next ( #53896 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: andyluo7 <andy.luo@amd.com >
Signed-off-by: lishunyang12 <lishunyang03@gmail.com >
Signed-off-by: Harry Huang <huanghaoyan.hhy@alibaba-inc.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: KungYork <firelikest@163.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: lishunyang12 <lishunyang03@gmail.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-31 05:57:56 +00:00
Hotragn Pettugani
687db59744
[Bugfix][Frontend] Truncate pooling prompts before padding them ( #54364 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
2026-08-30 22:12:03 -07:00
LironKesem
44fe2a392b
[CPU] add CPU support for Voxtral ( #53921 )
...
Signed-off-by: Liron Kesem <52330564+LironKesem@users.noreply.github.com >
2026-08-31 04:20:37 +00:00
Jee Jee Li
648b7468b8
[CI/Build] Fix Kimi K3 Eagle3 test fixture ( #54482 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-08-31 03:29:45 +00:00
Isotr0py
555ea65e8c
[Frontend] Add video embeds input support ( #54242 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-31 03:20:47 +00:00
Lucas Wilkinson
5707355209
[Bugfix][MLA] Fix BLHNC addressing for FlashInfer sparse MLA ( #54465 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-08-31 01:13:42 +00:00
AI-Infra Rookie
c92b29a1d4
[Bugfix] Make metadata send non-blocking in GroupCoordinator.isend_tensor_dict ( #49274 )
...
Signed-off-by: z00897177 <zhouxinyi6@huawei.com >
2026-08-30 23:51:25 +00:00
7ab2923489
[Flashinfer] Upgrade Flashinfer version to 0.6.18 ( #54313 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-08-30 16:01:05 -07:00
Kevin H. Luu
7a100bb617
[CI] Restore gpu_1_queue routing for torch-abi audit ( #54468 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-08-30 14:59:36 -07:00
Kevin H. Luu
b2dc864bb6
[Bugfix][Spec Decode] Keep default CUDA graph sizes memory-safe ( #54418 )
...
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com >
2026-08-30 18:02:31 +00:00
Aditya Jha and Claude
9d0fe9bac8
[Bugfix][Quantization][MoE] Route weight only NVFP4 checkpoints through W4A16 ( #54427 )
...
Signed-off-by: Aditya Jha <4adityajha@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-30 13:02:15 -04:00
Juan Pérez de Algaba
e79961c946
[codeowners] Add jperezdealgaba to security file ownership ( #54358 )
...
Adds Juan Pérez de Algaba (`@jperezdealgaba`) alongside Russell Bryant as a CODEOWNER for the vLLM security guide, security policy, and vulnerability-management documentation.
2026-08-30 16:24:21 +00:00
waizuichougou and OpenAI Codex
f6895a5fcb
[Bugfix][Multimodal] Avoid caching full prompts in fallback ( #54439 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-30 16:19:38 +00:00
56058fd572
[Bugfix][Kernel] Keep packed GDN decode beta in FP32 ( #53877 )
...
Signed-off-by: CherryLemon <lemon_cn@alumni.sjtu.edu.cn >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-08-30 22:15:36 +08:00
machero and Claude
87b9b5b8d9
[Test][VLM] Add batch-invariance tests for Qwen3-VL ( #53531 )
...
Signed-off-by: machero <sr1123640465@126.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-30 09:49:41 -04:00
Taneem Ibrahim
fe6db3ed5e
[Bugfix] Validate scale-out transfer params ( #54324 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-30 09:46:12 -04:00
Harry Mellor and Claude Opus 5
488b6da105
[Doc] Fix griffe warnings in HYV4 tool parser ( #54412 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-30 13:21:22 +00:00
Kevin H. Luu
5e71a11eb2
[CI] Mark L4 GPU test steps with device: l4 for EKS migration ( #54326 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-08-30 04:40:41 -07:00
Kevin H. Luu
78fa18910e
ci: add MIG slice size to H200 job labels ( #54420 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-08-30 04:35:01 -07:00
Jiangyun Zhu and OpenAI Codex
8c51b92654
[Bugfix] Avoid global config lookup in sparse indexer forward ( #54400 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-30 03:31:47 -07:00
2c7d7dd64a
[Perf][ROCm] Dual-stream decode with hipgraphs ( #52033 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-08-30 08:24:20 +00:00
Devon Krisman and Claude Fable 5
4f78a8fdd0
[Model] Honor cap_pixels_per_frame in Qwen3-VL memory profiling ( #54380 )
...
Signed-off-by: Devon Krisman <git@krisman.dev >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-30 08:07:29 +00:00
Canlin Guo and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
b383e16396
[Bugfix] Reset cached Mamba align metadata on profiling teardown ( #54044 )
...
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-30 00:43:20 -07:00
Kevin H. Luu and Kimi Code
8fa4c6cdb3
[CI] Add explicit step keys to 18 hardware test steps ( #54330 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-08-30 00:08:25 -07:00
Juan Pérez de Algaba
1dc464d426
[Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion ( #54353 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-08-30 04:28:16 +00:00
Yizhou
1ebff996a1
[Fix] Improve ROCm detection in WSL environments ( #38434 )
...
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com >
2026-08-30 04:23:10 +00:00
Connor Carpenter
7a67941c1d
[Rust Frontend][gRPC] Add audio and video media inputs ( #53760 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
2026-08-30 11:31:37 +08:00
b016ed8ea3
[Bugfix][Multimodal] Release Qwen2.5-VL and Qwen3-VL RoPE caches with the model ( #54346 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-30 11:10:29 +08:00
dbf662c9e8
[ROCm][MLA] Reach FULL cudagraphs for AITER MLA speculative decoding ( #51171 )
...
Signed-off-by: yudigege86 <naqin@amd.com >
Co-authored-by: yudigege86 <naqin@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-29 19:49:55 -07:00
hcl
b5707bf994
[Bugfix][MoE] Enable cuBLAS out_dtype router GEMM on all CUDA archs (fixes family-120/GB10) ( #54048 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
2026-08-29 19:18:42 -07:00
yzong-rh
680e2177e4
[CI][Ray] Fix flaky multi-node assignment test after placement-group teardown ( #53621 )
2026-08-29 23:48:22 +00:00
Oxana Korzh and Claude Opus 5
6cddad414e
[CI][Test] Deflake test_mem.py sleep-mode asserts via allocator bookeeping ( #54312 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-29 12:05:17 -07:00
Giancarlo Delfin
fe755c8899
[Bugfix][Model Runner V2][Spec Decode] Decouple the draft's gumbel noise stream from the target's ( #54282 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-08-29 09:58:29 -07:00
Kaif Kohari and Cyrus Leung
7fbfca2670
[Bugfix][Frontend] Only echo the assistant turn in batched chat completions ( #52529 )
...
Signed-off-by: Kaif <kaifkohari10@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-29 09:44:17 -07:00
jack and QwertyJack
5b0e5b69ac
[Bugfix][Frontend] Validate stop_token_ids against vocab size ( #54196 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
2026-08-29 16:25:35 +00:00
Shenglei Fu and Claude Fable 5
129087ddab
[Perf] Reuse topk SparseMatrix routing metadata in GPT-OSS MoE forward ( #45457 )
...
Signed-off-by: Shenglei Fu <sfu@confluent.io >
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-29 12:14:39 -04:00
Nicolò Lucchesi
7f4793eaa3
[Nixl][PD] DCP support for MLA models ( #50611 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-29 16:03:53 +00:00
Isotr0py
4fc943b867
[Multimodal] Deprecate PyAV video decoder backend ( #54231 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-08-29 15:13:28 +00:00
Taneem Ibrahim
fd5d3aea94
[Mypy] Fix typing for J models ( #54130 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-29 08:50:15 -04:00
Andreas Karatzas and OpenAI Codex
cacc429f62
[ROCm][CI] Stage E gating ( #50920 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-28 23:52:23 -07:00
d3d79ffc1e
[Bugfix][Spec Decode] Capture the widest uniform decode batch by default ( #50488 )
...
Signed-off-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com >
Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: Rahul Chalamala <rahul@modal.com >
Co-authored-by: Janelle Cai <janelle.cai@modal.com >
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <devin@cognition.ai >
2026-08-28 22:38:13 -07:00
a758a9f671
[Perf][KV Connector] Pin token_indices before non_blocking H2D in hf3fs helper ( #54293 )
...
Signed-off-by: Keyi Li <likey6688@gmail.com >
Co-authored-by: Keyi Li <likey6688@gmail.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
2026-08-28 22:20:17 -07:00
f4f3bcd4b4
[Test] Assert co-located RayExecutorV2 stores publish distinct ports ( #54310 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-28 22:13:31 -07:00
Oxana Korzh and Claude Opus 5
fb68025138
[Bugfix][MRV2] Release model and KV cache on in-process engine shutdown ( #54162 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-28 22:09:08 -07:00
Hotragn Pettugani
3958a420f0
[Bugfix][V1] Keep an encoder cache entry until its last occurrence is freed ( #54284 )
...
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in >
2026-08-29 05:08:01 +00:00
Summer Yang
6d4562c59b
[Attention][DCP] Enable FlashInfer MLA for DSpark drafting ( #54277 )
...
Signed-off-by: Summer Yang <girasoleyang@gmail.com >
2026-08-28 21:11:28 -07:00
