Commit Graph
22490 Commits
Author SHA1 Message Date
f5ed3d9c03 [Fast Start] Charge daemon-held weights against gpu_memory_utilization (#57298)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
2026-09-29 17:58:35 +00:00
Jung JiyuandMisha Goin cb4f016c5d [Bugfix] Fix standalone torch.compile cache loading after relocation (#52142)
Signed-off-by: jungjiyu <libraryofjiyu@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-09-29 12:53:20 -04:00
cd94b21264 [Bugfix][Frontend] Return 400 for malformed RL dev route bodies (#59236)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:46:56 +00:00
Isotr0py 31876d8ea1 [Bugfix] Fix TorchCodec audio IO correctness (#58364)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
2026-09-29 11:34:48 -05:00
stefankoncarevicandAndreas Karatzas c3a4a36c20 [CI][ROCm] Fix stale basic_correctness path in Model Runner V2 Distributed (#59201)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-29 16:32:04 +00:00
Ashraf BhuiyanandWentao Ye 2454b5a4f0 [Mypy] Fix mypy typing for Whisper models (#58254)
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-09-29 17:10:28 +01:00
f4917dadc8 [Bugfix][Quantization] Stop sleep(level=2) from zeroing compressed-tensors KV scales (#57163)
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
2026-09-29 15:34:15 +00:00
Giulio De Pasquale 8aaeef343a [Bugfix][MiMo] Declare embedding_fields so an EPD pair can serve images (#58938)
Signed-off-by: Giulio De Pasquale <git@depasquale.giugl.io>
2026-09-29 23:33:01 +08:00
ac7f3e11ea [Bugfix] Profile maximum DeepSeek V4.1 vision features (#57898)
Signed-off-by: ahmed xijiaat <52128022+xijiaat@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-29 15:21:27 +00:00
stefankoncarevic 741edeebee [Bugfix][CI] Assert the logger call the Anthropic merge warning makes (#59202)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
2026-09-29 06:27:34 -07:00
cd947f5c62 [MM] Fix compiled ViT attention output layouts (#58182)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai>
2026-09-29 06:27:03 -07:00
Shijin ZhangandChauncey d95d1dcfb9 [Bugfix][Frontend] Force reasoning mode for GLM-5.3 chat templates in the GLM MoE parser (#56994)
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-09-29 21:26:34 +08:00
dfc8e0f3e2 [Frontend] Support strict MiMo-V2.6 tool calling (#58019)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-29 12:47:47 +00:00
jackandQwertyJack 79a0307d74 [Config] Remove Engram CUDA-alike device restrictions (#59171)
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
2026-09-29 12:42:02 +00:00
Harry MellorandClaude Opus 5.5 998490cd9f [Docs] Speed up docs build ~5x (#58705)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 12:16:33 +01:00
Andreas KaratzasOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
05d89636fc [ROCm][CI] Mirror generic GEMM-RS/AR on MI355 (#58433)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 10:28:15 +00:00
3f3fbe286f [Bugfix][Rust Frontend] Account for new requests in DP routing (#58956)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-29 10:24:28 +00:00
Louie Tsai 208bb38c80 [Bugfix][Rust Frontend] Fix startup with config-only model (#58643)
Signed-off-by: louie-tsai <louie.tsai@intel.com>
2026-09-29 10:11:53 +00:00
af5b4857e1 [Fast Start] Support PP (#55477)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
2026-09-29 09:37:43 +00:00
yhcheong (vllmellm) 77e52645e9 [Bugfix][Model] MiMo: keep fused fp8 qkv_proj pairing state across weight-loading calls (#58142)
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
2026-09-29 09:37:20 +00:00
aoshen02andOpenAI Codex 4861833ae2 [Bugfix][Core] Allow AuxOutput reset after abort without running requests (#59060)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-29 09:23:32 +00:00
Chauncey 3e2a7e74a5 [Feat][Model] Enable KDA prefill checkpoints for GLM-5.3-Flash (#56960)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-29 09:14:30 +00:00
Andreas KaratzasandOpenAI Codex 3acddf01e7 [Bugfix][ROCm] Keep Mooncake bootstrap ports bound during startup (#58967)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-29 04:14:26 -05:00
Andreas Karatzas 6ebb5bd64f [ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX (#59137)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
2026-09-29 03:40:07 -05:00
Jakub ZakrzewskiandOpenAI Codex 70dc122f81 [MM] Enable device normalization for Llama Nemotron VL Embed/Rerank (#57928)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-29 08:33:52 +00:00
Chauncey 61349e342d [Misc] Avoid repeated warnings for merged Anthropic system messages (#59172)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-09-29 01:31:22 -07:00
Yifan QiaoandClaude Fable 5.1 4b2e1cfa7b [Model] Decoder-side SWA bounded replay for DeepSeek-V4.1 (#58132)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 08:25:46 +00:00
9af952c555 [Bugfix][Rust Frontend] Support raise_exception in chat templates (#59020)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yashasvi Asthana <yashasvi.asthana@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-29 08:01:52 +00:00
Jin TaoCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Shanshan Shen
77d9cda688 [ROCm][Model][Bugfix] Fix GLM-5.2 shared-expert fusion and MTP on the ROCm DSA path (#58904)
Signed-off-by: Jin Tao <jin.tao@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-29 07:57:23 +00:00
Juntian LiuandClaude Opus 5.5 30f5c01d4e [DSv4.1] Avoid runtime recompiles of _ring_slot_mapping_kernel (#59119)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 07:31:30 +00:00
Václav ČadekandCursor Agent 6cbbea8721 [Bugfix] Hoist $defs/definitions in Cohere parser tool schema composition (#49602)
Signed-off-by: Václav Čadek <vaclavcadek@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-09-29 07:11:56 +00:00
Andreas KaratzasandOpenAI Codex 78fc1e6274 [ROCm][CI] Add MI355 TP2 AR-RMS and B200 fusion mirrors (#58284)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-29 07:10:26 +00:00
Andreas KaratzasandOpenAI Codex 69d5c81fcd [ROCm][CI] Mirror split kernel groups on MI355 and fix exposed tests (#58654)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Signed-off-by: Codex <codex@openai.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-29 02:08:31 -05:00
Andreas KaratzasandCodex 0ae480ff1a [CI][ROCm] Mirror large-model GSM8K evaluations on MI355 (#58683)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-29 02:06:44 -05:00
Jakub ZakrzewskiOpenAI CodexOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
d86cf91240 [MM][Mistral3] Image preprocessing optimization (#57531)
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-29 06:53:38 +00:00
65ed0a2642 [Bugfix][GLM-5.3-Flash] Take video placeholder timestamps from the pixel path's frame sampler (#55647)
Signed-off-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: NNNtrance <232268405+NNNtrance@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
2026-09-29 06:45:29 +00:00
MOVZXandHinata ae0350c0de [Spec Decode][MRV2] RecoverSSM for Qwen GDN and PLE short conv (PR #58863)
Intake of open upstream PR vllm-project/vllm#58863. Active only with
VLLM_USE_V2_MODEL_RUNNER=1; inert otherwise.

Speculative decoding on Gated DeltaNet snapshotted the full recurrent
state after every draft position and reserved one Mamba block per draft
position. On a 24 GB card that is reserved VRAM we never use, plus
per-step copy traffic.

RecoverSSM, the protocol Kimi-K3 KDA already uses, replaces this:
verify runs from a read-only checkpoint and stores a small per-token
replay record [correction (V), normalized key (K), gate (1)] in fp32;
commit replays only the accepted tokens and writes the recurrent state
once, plus the block-boundary state in align mode. The PLE dilated
short conv compacts its accepted window with the same plan.

Conflict in vllm/models/qwen4_exp/nvidia/mtp.py: our get_top_tokens()
work put LocalArgmaxMixin on Qwen4ExpMTP, the PR adds SupportsReplaySSM
to the same base list. Kept both.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 13:39:09 +07:00
63b9931b73 [ROCm][DSv4.1] Paged MXFP4 sparse indexer on aiter's MQA-logits kernel (#58671)
Signed-off-by: Mehmet Cagri Kaymak <mehmet.kaymak@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
2026-09-29 14:37:57 +08:00
MOVZXandHinata a1d3c2cf1e [Spec Decode] Fused draft metadata updates for the QSA side cache (PR #58449)
Intake of open upstream PR vllm-project/vllm#58449.

With MTP num_speculative_tokens > 1, the V2 autoregressive speculator
reuses one attention-metadata build across draft steps only when every
group's builder sets supports_draft_decode_metadata_update. The QSA
side-cache builder (QWEN4_EXP_EXP_QSA_STATE) did not, so every draft
step rebuilt attention metadata for the whole model.

QSAMetadataBuilder now implements update_draft_decode_metadata: it
re-launches the kernel with in-place seq_lens updates, so one metadata
build serves every draft step.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 13:36:09 +07:00
MOVZXandHinata cb275fdf20 [Qwen4Exp] Fix MTP buffer placement, config parsing, Triton warmup (PR #56742)
Intake of open upstream PR vllm-project/vllm#56742. Every rig here runs
MTP on both Qwen3.8 models, so these three bugs sit on our hot path.

The MTP hidden buffer was allocated with torch.empty and no device, so
it followed the ambient default instead of device_config.device; on a
CPU-default process the draft buffer lands on the wrong device. Both
NVIDIA and AMD backends now call make_mtp_hidden_buffer() in
vllm/models/qwen4_exp/common/mtp.py, which pins it to the configured
device.

Config parsing now accepts configs carrying M-RoPE keys and the
Transformers spelling of layer_types; both previously failed at load.

Triton warmup: Qwen4Exp was missing from the Qwen warmup dispatch, so
its Triton kernels compiled during the first served requests instead of
at startup.

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 13:36:05 +07:00
Bugen ZhaoandClaude Opus 5.5 e05095c9e1 [Rust Frontend] Feed parser tests and benchmarks attributed input (#59048)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:23:54 +00:00
Bowen WangandClaude Opus 5.5 35d6fb3187 [Perf][Spec Decode] Enable fused multi-step draft decode for FlashInfer trtllm-gen (#58371)
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:06:47 +00:00
MOVZX a63dca4afb Merge remote-tracking branch 'upstream/main' into patches
Signed-off-by: MOVZX <movzx@yahoo.com>
2026-09-29 12:54:18 +07:00
Bugen ZhaoandClaude Opus 5.5 28f6739576 [Rust Frontend] Relax schema-aware tool argument conversion (#59004)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 05:40:41 +00:00
Ren YuzhouandClaude ec5e0c352f [Bugfix][Engram] Keep THP tables private when resolving shared memory (#59068)
Signed-off-by: Ren Yuzhou <54501155+yuzhouo7@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-29 05:19:49 +00:00
Shinichi Hemmi 17cc3cd80e [Model] Engine based plamo3 parser (#56575)
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp>
2026-09-29 05:17:42 +00:00
shaohuaxiandCodex 36768d1bfd [Bugfix][Frontend] Preserve caller parameters in offline pooling (#58842)
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-29 05:02:59 +00:00
Bowen WangandClaude Opus 5.5 1fde8ff7da [Bugfix] Detect the CUDA toolkit the way FlashInfer does in has_flashinfer() (#58360)
Signed-off-by: Bowen Wang <bowen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 04:14:49 +00:00
debanshd 13c8589b86 fix: return loaded parameters in Aria load_weights (#57476)
Signed-off-by: Debanshu Das <debanshu@google.com>
2026-09-29 04:05:07 +00:00
Daniel M. García-Ocaña Hernández 295a0b8d64 [Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution (#56372)
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com>
2026-09-29 03:58:39 +00:00