e4294144a8
[Perf] Keep non-speculative GDN decode on the standard path ( #59735 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
2026-10-02 13:44:53 +00:00
Jiangyun Zhu and Claude Opus 5.5
d648847fe4
[CI][Kimi-K3] Test prefix cache reuse with KV offload, P/D and DCP ( #59229 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 20:58:35 +08:00
Stefano Castagnetta and Claude Opus 5
f0b4b36890
[Bugfix][Quantization] Detect NVFP4 in ModelOpt mixed-precision checkpoints ( #56050 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-02 12:30:22 +00:00
Abhay Shukla and Claude Opus 5.5
fa212b8dc5
[Test] Re-enable Ovis2.5 and Ovis2.6-MoE vLLM tests on Transformers v5 ( #59763 )
...
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:59:40 +00:00
Harry Mellor and Claude Opus 5.5
5d20979fc9
[Misc] Remove code paths for Transformers < 5.16.1 ( #59762 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:17:24 +00:00
8c60714da5
[Bugfix][LoRA] Fall back for high-rank MoE LoRA ( #55161 )
...
Signed-off-by: Raina Zhong <yuweiz@nvidia.com >
Co-authored-by: Raina Zhong <yuweiz@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 13:59:28 +03:00
Simon Danielsson and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
98b29aa99a
[ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes ( #58167 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:47:41 +01:00
Simon Danielsson and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
751fa96a49
[ROCm][Perf][GLM-5.3-Flash] Remove redundant copy after ragged sparse MLA ( #58569 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:45:51 +01:00
5688a4dd4a
[Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling ( #59565 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-02 10:43:29 +00:00
af2a582d5c
[Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix ( #59293 )
...
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-02 11:21:20 +01:00
aoshen02 and Claude Opus 5.5
8e58ac22af
[Feature] Offload KV-init runtime state on sleep ( #59158 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 09:49:21 +00:00
Juan Pérez de Algaba
c15672cfb4
[Security] Accept zero-sum DeepSeek-OCR pixel tensors ( #59417 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-10-02 09:17:12 +00:00
9d2a6f52b2
[Rust Frontend] Expose gRPC port in Python vllm serve ( #59659 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 08:14:37 +00:00
kliuae
0154979656
[ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm ( #58344 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-10-02 06:58:04 +00:00
4056c8ac1f
[Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with a saturated GPU pool ( #59309 )
...
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-01 23:04:21 -07:00
f7999d2e44
[Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocates ( #59450 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-02 04:11:55 +00:00
Qiu Chunshuo and Lucas Wilkinson
8486f1ebfe
[Perf][PCP] Shard decode requests across PCP ranks ( #52162 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-01 23:50:13 -04:00
a810316c08
[Bugfix] Keep batch-invariance NCCL pins out of the weight-transfer group ( #59500 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-02 11:19:28 +08:00
Martin Hickey
fcfacc3499
[Bugfix][Frontend] Seed derender detokenization from the prompt ( #59046 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-02 03:17:56 +00:00
Prudhvi Vuda and OpenAI Codex
ea106db806
[Bugfix][Bench] Clean up synthetic video files and writers ( #59529 )
...
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-02 02:51:34 +00:00
9bd993564f
[Frontend] Parse tool calls and reasoning from checkpoint response templates ( #58604 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 02:33:00 +00:00
Hyunjae Woo and aoshen02
5a661487e2
[Feature] Add native ModelExpress weight transfer backend ( #58399 )
...
Signed-off-by: Hyunjae Woo <hwoo@nvidia.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-02 02:29:35 +00:00
Flora Feng
785b990116
[Distributed] Enable custom all-reduce under VLLM_BATCH_INVARIANT ( #58623 )
...
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-10-02 02:25:48 +00:00
Harry Mellor and Claude Opus 5
e3cae8d2ac
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor ( #57387 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-02 02:25:18 +00:00
aoshen02 and Claude Opus 5.5
866f02db8a
[Feature] Release WorkspaceManager scratch on sleep ( #59156 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 02:14:37 +00:00
4caa060053
[TEST][CI] Fix serve rlhf tests subprocess import errors ( #59622 )
...
Signed-off-by: microslaw <you@example.com >
Co-authored-by: microslaw <you@example.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-02 09:47:47 +08:00
d58da79de7
[Metrics][P/D] Add KV-fetch stage gauges for async KV loads ( #58874 )
...
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-02 01:24:33 +00:00
28c57456db
[Bugfix][Frontend] Accept Anthropic tool_addition and tool_removal content blocks ( #57693 )
...
Signed-off-by: Anurag Singh <dr.anuragkr@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Antigravity <antigravity@google.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-01 20:03:36 -04:00
Divakar Verma and Claude Sonnet 5
e6674262f1
[Bugfix][CI] Widen DBO+DP+EP GSM8K accuracy margin on ROCm ( #59700 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-01 23:37:36 +00:00
hcl and Harry Mellor
e35082e7b6
[Bugfix][CLI] Include inherited field docstrings in get_attr_docs ( #49821 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Signed-off-by: hclsys <chenglunhu@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-01 23:35:58 +00:00
Aarushi Jain
d2125ea681
[ROCm][CI] Raise the MI355 DeepSeek-R1 GSM8K startup wait to 1800s ( #59666 )
...
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com >
2026-10-01 22:30:13 +00:00
Flora Feng
a5105dba05
[Frontend] Port Step-3.5 parsers to the streaming parser engine ( #59321 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-10-01 22:18:37 +00:00
ca65eb67d9
[Perf][HiSparse] Avoid repeated prefix scans and residency updates ( #57930 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-10-01 21:48:18 +00:00
Yongye Zhu and Claude Opus 5.5
7566d83bd3
[Attention][MiniMax-M3] NVFP4 KV cache on the MSA sparse attention path ( #59300 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 14:39:55 -07:00
Zijing Liu
ea84051819
[KVConnector][NIXL] Count completion notifications that arrive after KV expiry ( #58875 )
...
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com >
2026-10-01 21:30:58 +00:00
2a52715ef7
[Bugfix][Responses] Use standard reasoning content-part events ( #59652 )
...
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-01 19:20:57 +00:00
Chanbin Lim and Chendi.Xue
47de9d4d04
[KV Connector][NIXL] Coalesce host-buffer KV copies across cache groups ( #54483 )
...
Signed-off-by: beenpow <57864041+beenpow@users.noreply.github.com >
Co-authored-by: Chendi.Xue <chendi.xue@intel.com >
2026-10-01 13:10:54 -05:00
e9be5323f0
[Perf] Integrate flash-maxsim Triton kernels for late-interaction scoring ( #40337 )
...
Signed-off-by: roi.pony <roi.pony@ibm.com >
Co-authored-by: roi.pony <roi.pony@ibm.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-10-01 14:06:43 -04:00
Matthew Bonanni and Claude Opus 5.5
3a69636645
[Bugfix][HiSparse] Fix a chunked-prefill preemption livelock ( #59494 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 13:02:31 -04:00
26bfdfb6bc
[ROCm][Perf] Parallelise AITER MLA page-index expansion over token chunks ( #57978 )
...
Signed-off-by: lifulu <fululi12@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 12:00:15 -05:00
Shijie Lyu
9aaef4296f
[Bugfix][Frontend] Check reused prompt token ids against the vocab before streaming ( #59555 )
...
Signed-off-by: Shijie Lyu <lshjhf@gmail.com >
2026-10-01 09:26:55 -07:00
aniskumar-nv and Cyrus Leung
83cadd65d9
[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it ( #53020 )
...
Signed-off-by: Anish Kumar <aniskumar@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-01 09:26:33 -07:00
b538d807ff
[Bugfix][MRV2] Keep GDN prefill checkpoint metadata local to each cache group ( #59536 )
...
Signed-off-by: Jingqiao Zhang <ai-jz@users.noreply.github.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Jingqiao Zhang <ai-jz@users.noreply.github.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-01 22:59:15 +08:00
5d9214d6b2
[Bugfix][Frontend] Strip x-anthropic-billing-header billing header from /v1/chatcompletions ( #59419 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-01 13:44:40 +00:00
cb6458c5ce
[Core] Remove deprecated mamba_cache_mode "all" ( #58997 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-10-01 02:59:37 -07:00
Mark McLoughlin and OpenAI Codex
91fcec1235
[Core][Logging] Add built-in JSON formatter ( #58739 )
...
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-01 09:11:46 +01:00
5addde2a0a
[CPU][Zen] Pass f32 weight scales to the zentorch INT8 MoE ( #59434 )
...
Signed-off-by: Ganesh R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-01 15:51:53 +08:00
jiangyunfan1 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e12b735109
[Test] Make Anthropic messages test compatible with SDK 1.x via extra_body ( #57780 )
...
Signed-off-by: jiangyunfan1 <jiangyunfan1@h-partners.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 07:38:29 +00:00
e12291d733
[CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers ( #54024 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Signed-off-by: Ganesh R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 15:18:25 +08:00
Wentao Ye
4d6874d137
[GLM 5.3 Perf] Skip qlnorm calculation for MHA, 4.4~7.7% E2E TTFT Improvement ( #58845 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-01 15:13:57 +08:00
