MOVZX
4a3ebc6b5b
Merge branch 'main' into patches
2026-10-05 08:21:46 +00:00
34051ad714
feat: Add video support for the Transformers backend ( #57441 )
...
pre-commit / pre-run-check (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-05 00:51:15 -07:00
042ab0305c
[Perf][Qwen4Exp] Merge QSA QKVG and indexer QK projections ( #59533 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: namgyu-youn <namgyu.dev@gmail.com >
2026-10-05 07:03:29 +00:00
MOVZX
f83ecfac16
Merge remote-tracking branch 'upstream/main' into patches
2026-10-05 08:52:31 +07:00
Vadim Gimpelson and Claude
4f52fa35ef
[Bugfix] Fix Qwen4Exp PLE embedding rejecting INC (AutoRound) checkpoints ( #59990 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-05 01:52:06 +00:00
Harry Mellor
9e0b6fcf77
[CI] Bump Transformers version to 5.18.0 ( #59621 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-04 00:42:35 +00:00
MOVZX
85c31d47ad
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 22:06:10 +07:00
MOVZX
2877121b2d
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-03 21:59:03 +07:00
Markus Hartikainen
b0e21b3083
[ROCm][Kimi-K3] Add opt-in gfx942 MXFP4-to-int4 conversion ( #51274 )
...
Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com >
Signed-off-by: Hartikainen <mahartik@amd.com >
2026-10-03 14:28:02 +08:00
Julien Debache
111f71a6db
[feat] FlashInfer CuteDSL MegaMoE integration ( #54049 )
...
Signed-off-by: jdebache <jdebache@nvidia.com >
2026-10-02 15:43:33 -04:00
Thang Nguyen and Claude Opus 5.5
7cfd233538
Revert "[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it" ( #53020 ) ( #59805 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 10:19:25 -07:00
Jiangyun Zhu and Claude Opus 5.5
d648847fe4
[CI][Kimi-K3] Test prefix cache reuse with KV offload, P/D and DCP ( #59229 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 20:58:35 +08:00
Abhay Shukla and Claude Opus 5.5
fa212b8dc5
[Test] Re-enable Ovis2.5 and Ovis2.6-MoE vLLM tests on Transformers v5 ( #59763 )
...
Signed-off-by: AbhayShuklaIIT <abhay.shukla@live.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:59:40 +00:00
Harry Mellor and Claude Opus 5.5
5d20979fc9
[Misc] Remove code paths for Transformers < 5.16.1 ( #59762 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:17:24 +00:00
Simon Danielsson and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
98b29aa99a
[ROCm][Perf][GLM-5.3-Flash] Add AITER topk backend for decodes ( #58167 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-02 11:47:41 +01:00
5688a4dd4a
[Bugfix][GLM-5.3] Size the image encoder cache from the exact token ceiling ( #59565 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-10-02 10:43:29 +00:00
Juan Pérez de Algaba
c15672cfb4
[Security] Accept zero-sum DeepSeek-OCR pixel tensors ( #59417 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-10-02 09:17:12 +00:00
kliuae
0154979656
[ROCm][Perf] Kimi-K3 enable prefill checkpoints on ROCm ( #58344 )
...
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com >
2026-10-02 06:58:04 +00:00
Harry Mellor and Claude Opus 5
e3cae8d2ac
[Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor ( #57387 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-02 02:25:18 +00:00
aniskumar-nv and Cyrus Leung
83cadd65d9
[Bugfix] Tie lm_head.weight for Nemotron Parse when checkpoint omits it ( #53020 )
...
Signed-off-by: Anish Kumar <aniskumar@nvidia.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-01 09:26:33 -07:00
Shuolei Wang and Thien
c9578f10ab
[Qwen3.8-Flash-Next] Fuse main QK-norm/RoPE/gate and KV-cache write into the QSA pre-indexer launch ( #57097 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Thien <thien@inferact.ai >
Co-authored-by: Thien <thien@inferact.ai >
2026-10-01 14:21:06 +08:00
766b4ae54c
[Kimi-K3] Add FlashInfer speculative KDA backend ( #54255 )
...
Signed-off-by: Duncan Moss <dmoss@nvidia.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-01 02:28:50 +00:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
Yida Weng and Grok
17e9295dd5
[BugFix][Multimodal] Pick worst-case DeepSeek-V4 VL dummy image size ( #59271 ) ( #59373 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Co-authored-by: Grok <grok@x.ai >
2026-09-30 15:09:11 +00:00
5225f1bacb
[MyPy][3/N] Fix MyPy errors in test groups (part 3) ( #55939 )
...
Signed-off-by: Taimys <ffy2281693949@gmail.com >
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Co-authored-by: Taimys <ffy2281693949@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 11:06:07 +00:00
Mathew Odden and opencode+glm-5.3+vllm
91dab0eb7f
[ROCm][Perf] Allocate the pinned PLE prefetch buffer lazily ( #58797 )
...
Signed-off-by: Mathew Odden <modden@redhat.com >
Co-authored-by: opencode+glm-5.3+vllm <opencode+glm-5.3+vllm@example.com >
2026-09-30 17:43:16 +08:00
yhcheong (vllmellm)
16d4ac4ba3
[ROCm][MoE] Support MiMo-V2.6 MXFP4 on gfx942 ( #58262 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-09-30 17:23:00 +08:00
Simon Danielsson
3627a6a124
[ROCm][Perf][GLM-5.3-Flash] Stride-aware decode KDA ( #57979 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-30 08:16:29 +00:00
cjackal
0a30bc3f9a
[Model] Extend device-side mm normalization to GLM4V/GLM5Next ( #55389 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-09-30 15:44:46 +08:00
5faf81a429
[CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel ( #58268 )
...
Signed-off-by: Harshal Adhav <harshal.adhav@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 13:52:53 +08:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
834ad456d3
[Refactor] Remove dead tests code ( #58916 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-30 04:39:03 +00:00
Chauncey
3e2a7e74a5
[Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash ( #56960 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-29 09:14:30 +00:00
Jakub Zakrzewski and OpenAI Codex
70dc122f81
[MM] Enable device normalization for Llama Nemotron VL Embed/Rerank ( #57928 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-29 08:33:52 +00:00
Yifan Qiao and Claude Fable 5.1
4b2e1cfa7b
[Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 ( #58132 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-29 08:25:46 +00:00
d86cf91240
[MM][Mistral3] Image preprocessing optimization ( #57531 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 06:53:38 +00:00
65ed0a2642
[Bugfix][GLM-5.3-Flash] Take video placeholder timestamps from the pixel path's frame sampler ( #55647 )
...
Signed-off-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 06:45:29 +00:00
MOVZX and Hinata
ae0350c0de
[Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863 )
...
Intake of open upstream PR vllm-project/vllm#58863 . Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.
Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.
RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.
Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:39:09 +07:00
MOVZX and Hinata
a1d3c2cf1e
[Spec Decode] Fused draft metadata updates for the QSA side cache (PR #58449 )
...
Intake of open upstream PR vllm-project/vllm#58449 .
With MTP num_speculative_tokens > 1, the V2 autoregressive speculator
reuses one attention-metadata build across draft steps only when every
group's builder sets supports_draft_decode_metadata_update. The QSA
side-cache builder (QWEN4_EXP_EXP_QSA_STATE) did not, so every draft
step rebuilt attention metadata for the whole model.
QSAMetadataBuilder now implements update_draft_decode_metadata: it
re-launches the kernel with in-place seq_lens updates, so one metadata
build serves every draft step.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:36:09 +07:00
MOVZX and Hinata
cb275fdf20
[Qwen4Exp] Fix MTP buffer placement, config parsing, Triton warmup (PR #56742 )
...
Intake of open upstream PR vllm-project/vllm#56742 . Every rig here runs
MTP on both Qwen3.8 models, so these three bugs sit on our hot path.
The MTP hidden buffer was allocated with torch.empty and no device, so
it followed the ambient default instead of device_config.device; on a
CPU-default process the draft buffer lands on the wrong device. Both
NVIDIA and AMD backends now call make_mtp_hidden_buffer() in
vllm/models/qwen4_exp/common/mtp.py, which pins it to the configured
device.
Config parsing now accepts configs carrying M-RoPE keys and the
Transformers spelling of layer_types; both previously failed at load.
Triton warmup: Qwen4Exp was missing from the Qwen warmup dispatch, so
its Triton kernels compiled during the first served requests instead of
at startup.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:36:05 +07:00
MOVZX
a63dca4afb
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 12:54:18 +07:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00
cjackal and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
491f44adfa
[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next ( #51289 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 11:44:54 +08:00
Ashraf Bhuiyan and Wentao Ye
2619b6b72b
[Mypy] Fix mypy typing for Voxtral and vision models ( #58251 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-28 15:52:50 -04:00
xaguilar-amd and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a7ff44355c
[ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP ( #54956 )
...
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 10:31:29 -04:00
Jiangyun Zhu and OpenAI Codex
20b52e9f5b
[Perf][Qwen3.8] Reduce PLE metadata construction overhead ( #58114 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 20:46:22 +08:00
Thien Tran
d6cce94fd4
[Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA ( #58957 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-28 18:51:38 +08:00
a2be4d3cc7
[Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views ( #58961 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 08:11:46 +00:00
MOVZX
76af434852
Merge remote-tracking branch 'upstream/main' into patches
2026-09-28 12:06:52 +07:00
Kevin H. Luu and Claude Opus 5.5
0da126676e
[Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) ( #58900 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 04:52:51 +00:00
dd3bc9c72f
[Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups ( #58762 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Xin Yang <xyangx@amazon.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-28 04:36:58 +00:00
MOVZX
