MOVZX and Hinata
7e5f658d75
[Qwen4Exp] Pipeline-stage ownership for PLE, the MTP drafter and the vision tower
...
Each pipeline stage builds only the parts it owns: PLE when every PLE layer
lives on stage 0, MTP under PP (port of vLLM PR #46994 minus its ROCm MLA file
and tests) with the drafter on the last stage, and the vision tower on the
first stage only.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-09 19:43:53 +07:00
MOVZX
4a3ebc6b5b
Merge branch 'main' into patches
2026-10-05 08:21:46 +00:00
34051ad714
feat: Add video support for the Transformers backend ( #57441 )
...
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-05 00:51:15 -07:00
MOVZX
f83ecfac16
Merge remote-tracking branch 'upstream/main' into patches
2026-10-05 08:52:31 +07:00
Harry Mellor
9e0b6fcf77
[CI] Bump Transformers version to 5.18.0 ( #59621 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-04 00:42:35 +00:00
MOVZX
85c31d47ad
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 22:06:10 +07:00
MOVZX
2877121b2d
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 21:59:03 +07:00
C
44198f577f
[Bugfix][Model] Fix M-RoPE offset double-count in Qwen3-Omni ( #58890 )
...
Signed-off-by: C1ves <3121985758@qq.com >
2026-10-03 09:00:51 +08:00
Thang Nguyen and Claude Opus 5.5
7cfd233538
Revert "[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it" ( #53020 ) ( #59805 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 10:19:25 -07:00
Harry Mellor and Claude Opus 5.5
6e4efafc6e
[Model] Migrate GPT-NeoX, Phi, Seed-OSS and Jais2 to the Transformers modeling backend ( #59701 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:18:35 +00:00
Harry Mellor and Claude Opus 5.5
5d20979fc9
[Misc] Remove code paths for Transformers < 5.16.1 ( #59762 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:17:24 +00:00
af2a582d5c
[Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix ( #59293 )
...
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-02 11:21:20 +01:00
Juan Pérez de Algaba
c15672cfb4
[Security] Accept zero-sum DeepSeek-OCR pixel tensors ( #59417 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-10-02 09:17:12 +00:00
0cbac6cd13
[Model][Spec Decode] Enable EAGLE3/DSpark pipeline parallelism for Sarvam MLA ( #55902 )
...
Signed-off-by: mohit-sarvam <mohit@sarvam.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-02 04:28:19 +00:00
Harry Mellor and Claude Opus 5.5
8faacd707f
[Model] Migrate Glm, Arcee, CWM and Mellum to the Transformers modeling backend ( #59679 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 03:15:14 +00:00
Harry Mellor and Claude Opus 5.5
7b44aad5db
[MyPy] Fix mypy errors in vllm/model_executor/models/[kK]* ( #59402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 19:32:27 +00:00
aniskumar-nv and Cyrus Leung
83cadd65d9
[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it ( #53020 )
...
Signed-off-by: Anish Kumar <aniskumar@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-01 09:26:33 -07:00
cb6458c5ce
[Core] Remove deprecated mamba_cache_mode "all" ( #58997 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com >
2026-10-01 02:59:37 -07:00
c4fc3c7d32
[Bugfix][Spec Decode] Per-module LM heads for multi-layer MTP on Model Runner V2 ( #58921 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: chanh <channguyen@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 01:09:14 -07:00
e12291d733
[CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers ( #54024 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Signed-off-by: Ganesh R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 15:18:25 +08:00
Ning Xie
a37c402f32
[weight loader] add dtype equality validation between parameter and weight ( #51792 )
...
Signed-off-by: Andy Xie <andy.xning@gmail.com >
2026-10-01 03:07:31 +00:00
Andrii Skliar and Andrii Skliar
deb148e730
[Mamba] Add FlashInfer ReplaySSM support for MTP ( #52928 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-09-30 21:41:17 +00:00
Varun Sundar Rabindranath
5463fe4962
[KV-Offloading][TP] : Expand replicated_layout detection to multi-group MLA ( #57652 )
2026-09-30 09:11:03 -04:00
effdea9a53
[MM][Mistral] Add compile support for Pixtral vision encoders ( #57168 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-30 20:27:12 +08:00
df5668e5c9
[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp ( #57172 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-30 12:19:33 +00:00
Avishek Goswami and Avishek Goswami
fb91712b38
[Model] Make the BERT/RoBERTa embedding class a class attribute ( #59348 )
...
Signed-off-by: Avishek Goswami <avishek.goswami@ibm.com >
Signed-off-by: Avishek Goswami <86944690+GOavi101@users.noreply.github.com >
Co-authored-by: Avishek Goswami <avishek.goswami@ibm.com >
2026-09-30 08:57:21 +00:00
Alexandra Sidorova
b81984879a
[K2 Horizon] fold partial-RoPE permutation into q/k (and norm) weights ( #55335 )
...
Signed-off-by: Alexandra Sidorova <asidorov@amd.com >
2026-09-30 16:03:37 +08:00
cjackal
0a30bc3f9a
[Model] Extend device-side mm normalization to GLM4V/GLM5Next ( #55389 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-09-30 15:44:46 +08:00
5faf81a429
[CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel ( #58268 )
...
Signed-off-by: Harshal Adhav <harshal.adhav@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 13:52:53 +08:00
Fabian Joswig and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1117140edb
[Bugfix][Model] Fix DiffusionGemma silently freezing attention mask under CUDA graph replay ( #51994 )
...
Signed-off-by: Fabian Joswig <fjosw@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-30 05:34:26 +00:00
a8e069f81b
[Bugfix][MiniMax M3] Share target embeddings with MTP under PP ( #58648 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-09-29 20:04:10 -07:00
9a190f142c
[Mypy] Fix mypy typing for Zamba2 models ( #58255 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-29 18:13:52 +00:00
Ashraf Bhuiyan and Wentao Ye
2454b5a4f0
[Mypy] Fix mypy typing for Whisper models ( #58254 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-29 17:10:28 +01:00
Giulio De Pasquale
8aaeef343a
[Bugfix][MiMo] Declare embedding_fields so an EPD pair can serve images ( #58938 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
2026-09-29 23:33:01 +08:00
yhcheong (vllmellm)
77e52645e9
[Bugfix][Model] MiMo: keep fused fp8 qkv_proj pairing state across weight-loading calls ( #58142 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-09-29 09:37:20 +00:00
Jakub Zakrzewski and OpenAI Codex
70dc122f81
[MM] Enable device normalization for Llama Nemotron VL Embed/Rerank ( #57928 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-29 08:33:52 +00:00
d86cf91240
[MM][Mistral3] Image preprocessing optimization ( #57531 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 06:53:38 +00:00
MOVZX and Hinata
ae0350c0de
[Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863 )
...
Intake of open upstream PR vllm-project/vllm#58863 . Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.
Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.
RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.
Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:39:09 +07:00
MOVZX
a63dca4afb
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 12:54:18 +07:00
debanshd
13c8589b86
fix: return loaded parameters in Aria load_weights ( #57476 )
...
Signed-off-by: Debanshu Das <debanshu@google.com >
2026-09-29 04:05:07 +00:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00
cjackal and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
491f44adfa
[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next ( #51289 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 11:44:54 +08:00
Yan Ma and Kunshang Ji
754aa29c00
[XPU] use rms_norm xpu kernel for context-key normalization ( #58896 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-29 07:09:58 +08:00
Ashraf Bhuiyan and Wentao Ye
2619b6b72b
[Mypy] Fix mypy typing for Voxtral and vision models ( #58251 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-28 15:52:50 -04:00
cd63bf9996
[Bugfix] Use the correct repository revision for secondary artifact loaders ( #57461 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 14:25:40 +00:00
Isotr0py
522101f6ed
[Multimodal] Avoid extra d2d for encoder cudagraph with fused input norm ( #56711 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-28 21:04:01 +08:00
Yannick Schnider
8cc9aa5ad3
[Bugfix][Model] Gemma4: register aliased embedding scalars as buffers ( #54213 )
...
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com >
2026-09-28 10:56:31 +00:00
zhao, zhenhui and Li, Jiang <jiang1.li@intel.com>
1668e5976d
[CPU] Use accelerator memory API in DiffusionGemma ( #58964 )
...
Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-28 13:31:42 +08:00
MOVZX
76af434852
Merge remote-tracking branch 'upstream/main' into patches
2026-09-28 12:06:52 +07:00
Kevin H. Luu and Claude Opus 5.5
0da126676e
[Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) ( #58900 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 04:52:51 +00:00
MOVZXandHinata
