f5ed3d9c03
[Fast Start] Charge daemon-held weights against gpu_memory_utilization ( #57298 )
...
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Signed-off-by: Xun Sun <UNIDY2002@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Xun Sun <UNIDY2002@outlook.com >
2026-09-29 17:58:35 +00:00
Jung Jiyu and Misha Goin
cb4f016c5d
[Bugfix] Fix standalone torch.compile cache loading after relocation ( #52142 )
...
Signed-off-by: jungjiyu <libraryofjiyu@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-29 12:53:20 -04:00
cd94b21264
[Bugfix][Frontend] Return 400 for malformed RL dev route bodies ( #59236 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 16:46:56 +00:00
Isotr0py
31876d8ea1
[Bugfix] Fix TorchCodec audio IO correctness ( #58364 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 11:34:48 -05:00
stefankoncarevic and Andreas Karatzas
c3a4a36c20
[CI][ROCm] Fix stale basic_correctness path in Model Runner V2 Distributed ( #59201 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-29 16:32:04 +00:00
Ashraf Bhuiyan and Wentao Ye
2454b5a4f0
[Mypy] Fix mypy typing for Whisper models ( #58254 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-29 17:10:28 +01:00
f4917dadc8
[Bugfix][Quantization] Stop sleep(level=2) from zeroing compressed-tensors KV scales ( #57163 )
...
Signed-off-by: AlanFokCo <alanfok2868@gmail.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-29 15:34:15 +00:00
Giulio De Pasquale
8aaeef343a
[Bugfix][MiMo] Declare embedding_fields so an EPD pair can serve images ( #58938 )
...
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io >
2026-09-29 23:33:01 +08:00
ac7f3e11ea
[Bugfix] Profile maximum DeepSeek V4.1 vision features ( #57898 )
...
Signed-off-by: ahmed xijiaat <52128022+xijiaat@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 15:21:27 +00:00
stefankoncarevic
741edeebee
[Bugfix][CI] Assert the logger call the Anthropic merge warning makes ( #59202 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-09-29 06:27:34 -07:00
cd947f5c62
[MM] Fix compiled ViT attention output layouts ( #58182 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai >
2026-09-29 06:27:03 -07:00
Shijin Zhang and Chauncey
d95d1dcfb9
[Bugfix][Frontend] Force reasoning mode for GLM-5.3 chat templates in the GLM MoE parser ( #56994 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-29 21:26:34 +08:00
dfc8e0f3e2
[Frontend] Support strict MiMo-V2.6 tool calling ( #58019 )
...
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Ubospica <ubospica@gmail.com >
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-29 12:47:47 +00:00
jack and QwertyJack
79a0307d74
[Config] Remove Engram CUDA-alike device restrictions ( #59171 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
2026-09-29 12:42:02 +00:00
Harry Mellor and Claude Opus 5.5
998490cd9f
[Docs] Speed up docs build ~5x ( #58705 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 12:16:33 +01:00
05d89636fc
[ROCm][CI] Mirror generic GEMM-RS/AR on MI355 ( #58433 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 10:28:15 +00:00
3f3fbe286f
[Bugfix][Rust Frontend] Account for new requests in DP routing ( #58956 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-29 10:24:28 +00:00
Louie Tsai
208bb38c80
[Bugfix][Rust Frontend] Fix startup with config-only model ( #58643 )
...
Signed-off-by: louie-tsai <louie.tsai@intel.com >
2026-09-29 10:11:53 +00:00
af5b4857e1
[Fast Start] Support PP ( #55477 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Xun Sun <UNIDY2002@outlook.com >
2026-09-29 09:37:43 +00:00
yhcheong (vllmellm)
77e52645e9
[Bugfix][Model] MiMo: keep fused fp8 qkv_proj pairing state across weight-loading calls ( #58142 )
...
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com >
2026-09-29 09:37:20 +00:00
aoshen02 and OpenAI Codex
4861833ae2
[Bugfix][Core] Allow AuxOutput reset after abort without running requests ( #59060 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-29 09:23:32 +00:00
Chauncey
3e2a7e74a5
[Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash ( #56960 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-29 09:14:30 +00:00
Andreas Karatzas and OpenAI Codex
3acddf01e7
[Bugfix][ROCm] Keep Mooncake bootstrap ports bound during startup ( #58967 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-29 04:14:26 -05:00
Andreas Karatzas
6ebb5bd64f
[ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX ( #59137 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
2026-09-29 03:40:07 -05:00
Jakub Zakrzewski and OpenAI Codex
70dc122f81
[MM] Enable device normalization for Llama Nemotron VL Embed/Rerank ( #57928 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-29 08:33:52 +00:00
Chauncey
61349e342d
[Misc] Avoid repeated warnings for merged Anthropic system messages ( #59172 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-29 01:31:22 -07:00
Yifan Qiao and Claude Fable 5.1
4b2e1cfa7b
[Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 ( #58132 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-29 08:25:46 +00:00
9af952c555
[Bugfix][Rust Frontend] Support raise_exception in chat templates ( #59020 )
...
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Yashasvi Asthana <yashasvi.asthana@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-29 08:01:52 +00:00
77d9cda688
[ROCm][Model][Bugfix] Fix GLM-5.2 shared-expert fusion and MTP on the ROCm DSA path ( #58904 )
...
Signed-off-by: Jin Tao <jin.tao@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-29 07:57:23 +00:00
Juntian Liu and Claude Opus 5.5
30f5c01d4e
[DSv4.1] Avoid runtime recompiles of _ring_slot_mapping_kernel ( #59119 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-29 07:31:30 +00:00
Václav Čadek and Cursor Agent
6cbbea8721
[Bugfix] Hoist $defs/definitions in Cohere parser tool schema composition ( #49602 )
...
Signed-off-by: Václav Čadek <vaclavcadek@gmail.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-09-29 07:11:56 +00:00
Andreas Karatzas and OpenAI Codex
78fc1e6274
[ROCm][CI] Add MI355 TP2 AR-RMS and B200 fusion mirrors ( #58284 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-29 07:10:26 +00:00
Andreas Karatzas and OpenAI Codex
69d5c81fcd
[ROCm][CI] Mirror split kernel groups on MI355 and fix exposed tests ( #58654 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Signed-off-by: Codex <codex@openai.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-29 02:08:31 -05:00
Andreas Karatzas and Codex
0ae480ff1a
[CI][ROCm] Mirror large-model GSM8K evaluations on MI355 ( #58683 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 02:06:44 -05:00
d86cf91240
[MM][Mistral3] Image preprocessing optimization ( #57531 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 06:53:38 +00:00
65ed0a2642
[Bugfix][GLM-5.3-Flash] Take video placeholder timestamps from the pixel path's frame sampler ( #55647 )
...
Signed-off-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-29 06:45:29 +00:00
MOVZX and Hinata
ae0350c0de
[Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863 )
...
Intake of open upstream PR vllm-project/vllm#58863 . Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.
Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.
RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.
Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:39:09 +07:00
63b9931b73
[ROCm][DSv4.1] Paged MXFP4 sparse indexer on aiter's MQA-logits kernel ( #58671 )
...
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-29 14:37:57 +08:00
MOVZX and Hinata
a1d3c2cf1e
[Spec Decode] Fused draft metadata updates for the QSA side cache (PR #58449 )
...
Intake of open upstream PR vllm-project/vllm#58449 .
With MTP num_speculative_tokens > 1, the V2 autoregressive speculator
reuses one attention-metadata build across draft steps only when every
group's builder sets supports_draft_decode_metadata_update. The QSA
side-cache builder (QWEN4_EXP_EXP_QSA_STATE) did not, so every draft
step rebuilt attention metadata for the whole model.
QSAMetadataBuilder now implements update_draft_decode_metadata: it
re-launches the kernel with in-place seq_lens updates, so one metadata
build serves every draft step.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:36:09 +07:00
MOVZX and Hinata
cb275fdf20
[Qwen4Exp] Fix MTP buffer placement, config parsing, Triton warmup (PR #56742 )
...
Intake of open upstream PR vllm-project/vllm#56742 . Every rig here runs
MTP on both Qwen3.8 models, so these three bugs sit on our hot path.
The MTP hidden buffer was allocated with torch.empty and no device, so
it followed the ambient default instead of device_config.device; on a
CPU-default process the draft buffer lands on the wrong device. Both
NVIDIA and AMD backends now call make_mtp_hidden_buffer() in
vllm/models/qwen4_exp/common/mtp.py, which pins it to the configured
device.
Config parsing now accepts configs carrying M-RoPE keys and the
Transformers spelling of layer_types; both previously failed at load.
Triton warmup: Qwen4Exp was missing from the Qwen warmup dispatch, so
its Triton kernels compiled during the first served requests instead of
at startup.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 13:36:05 +07:00
Bugen Zhao and Claude Opus 5.5
e05095c9e1
[Rust Frontend] Feed parser tests and benchmarks attributed input ( #59048 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 06:23:54 +00:00
Bowen Wang and Claude Opus 5.5
35d6fb3187
[Perf][Spec Decode] Enable fused multi-step draft decode for FlashInfer trtllm-gen ( #58371 )
...
Signed-off-by: Bowen Wang <bowen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 06:06:47 +00:00
MOVZX
a63dca4afb
Merge remote-tracking branch 'upstream/main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-29 12:54:18 +07:00
Bugen Zhao and Claude Opus 5.5
28f6739576
[Rust Frontend] Relax schema-aware tool argument conversion ( #59004 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 05:40:41 +00:00
Ren Yuzhou and Claude
ec5e0c352f
[Bugfix][Engram] Keep THP tables private when resolving shared memory ( #59068 )
...
Signed-off-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-29 05:19:49 +00:00
Shinichi Hemmi
17cc3cd80e
[Model] Engine based plamo3 parser ( #56575 )
...
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp >
2026-09-29 05:17:42 +00:00
shaohuaxi and Codex
36768d1bfd
[Bugfix][Frontend] Preserve caller parameters in offline pooling ( #58842 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 05:02:59 +00:00
Bowen Wang and Claude Opus 5.5
1fde8ff7da
[Bugfix] Detect the CUDA toolkit the way FlashInfer does in has_flashinfer() ( #58360 )
...
Signed-off-by: Bowen Wang <bowen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 04:14:49 +00:00
debanshd
13c8589b86
fix: return loaded parameters in Aria load_weights ( #57476 )
...
Signed-off-by: Debanshu Das <debanshu@google.com >
2026-09-29 04:05:07 +00:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00