Shanshan Shen
7ba3df63cb
[ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to ROCm platform ( #58983 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-28 05:46:28 -07:00
Jiangyun Zhu and OpenAI Codex
20b52e9f5b
[Perf][Qwen3.8] Reduce PLE metadata construction overhead ( #58114 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 20:46:22 +08:00
BaoYunkai
f6877f563c
[Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek V4 MTP drafter ( #57568 )
...
Signed-off-by: BaoYunkai <ybao@amd.com >
2026-09-28 20:16:38 +08:00
Yannick Schnider
8cc9aa5ad3
[Bugfix][Model] Gemma4: register aliased embedding scalars as buffers ( #54213 )
...
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com >
2026-09-28 10:56:31 +00:00
Thien Tran
d6cce94fd4
[Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA ( #58957 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-09-28 18:51:38 +08:00
Kevin H. Luu and Claude Opus 5.5
252279314b
Revert "[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs ( #58652 )" ( #59011 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 03:51:17 -07:00
Louie Tsai
ccfd1cea75
[CPU] Include vLLM Recipes tooling in release image to deploy models using vLLM Recipes ( #58796 )
...
Signed-off-by: louie-tsai <louie.tsai@intel.com >
2026-09-28 16:46:53 +08:00
a2be4d3cc7
[Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key views ( #58961 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 08:11:46 +00:00
Thang Nguyen and Claude Opus 5.5
66d2d4b2e7
[CI] Split Dynamic Shapes out of (H200 MIG 35GB) PyTorch Compilation + (MI300) mirror ( #58451 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 00:32:11 -07:00
413ae4b167
[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs ( #58652 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: khluu <khluu000@gmail.com >
2026-09-28 00:21:08 -07:00
004e37ece2
[Misc] Name each backend and its kernel block sizes in block-size errors ( #58557 )
...
Signed-off-by: Eugenio "Jay" Zuccarelli <11176606+jayzuccarelli@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-28 14:33:18 +08:00
Thien Tran and Kimi
4694145045
[Skills] Update kernel-microbenchmark to include ROCm ( #58646 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Kimi <noreply@moonshot.cn >
2026-09-28 14:18:38 +08:00
Rehan Khan and Li, Jiang <jiang1.li@intel.com>
fd2f3fcfa5
[CPU] Add video inferencing via torchcodec on s390x ( #58693 )
...
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-28 14:16:59 +08:00
zhao, zhenhui and Li, Jiang <jiang1.li@intel.com>
1668e5976d
[CPU] Use accelerator memory API in DiffusionGemma ( #58964 )
...
Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-28 13:31:42 +08:00
Turner Jabbour and Claude Sonnet 5
2407f405b5
[CI] Allowlist-shrink batch 1: wire 17 root-level tests + drop 5 stale watermarking entries into misc.yaml ( #58055 )
...
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-09-27 22:23:46 -07:00
MOVZX
76af434852
Merge remote-tracking branch 'upstream/main' into patches
2026-09-28 12:06:52 +07:00
af7f9488c2
[Benchmark] Add Responses API backend to vllm bench serve ( #54628 )
...
Signed-off-by: QHarshil <harshil_c@hotmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-28 05:04:11 +00:00
Kevin H. Luu and Claude Opus 5.5
0da126676e
[Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) ( #58900 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 04:52:51 +00:00
dd3bc9c72f
[Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups ( #58762 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Xin Yang <xyangx@amazon.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-09-28 04:36:58 +00:00
Thien Tran and Codex
55de40a2fc
[Minimax-M3] Add Encoder CUDA graph support ( #58673 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Codex <codex@openai.com >
2026-09-28 04:17:36 +00:00
Flora Feng
6f65fde419
[Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions ( #58958 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 04:12:04 +00:00
jz-yolo and jz-yolo
4a190dc518
[Model] Enable LoRA support for RobertaForSequenceClassification ( #58884 )
...
Signed-off-by: jz-yolo <jz-yolo@users.noreply.github.com >
Signed-off-by: Jane Zhu <jane.zhu@slack-corp.com >
Co-authored-by: jz-yolo <jz-yolo@users.noreply.github.com >
2026-09-28 03:50:12 +00:00
Andreas Karatzas
77fbd9e225
[Test][ROCm] Stabilize the mixed OLMoE LoRA test ( #58945 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
2026-09-28 11:47:41 +08:00
zhao, zhenhui and jiang1.li
5eaa50f151
[CPU] Build CPU wheels on Ubuntu 22.04 with AMX-FP8 support ( #58515 )
...
Signed-off-by: zhejiangxiaomai <zhenhui.zhao@intel.com >
Signed-off-by: jiang1.li <jiang1.li@intel.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
2026-09-28 11:46:02 +08:00
Tianyu Guo
53d2e16a97
[Bugfix][EPD] Skip sampling for encoder-only async steps ( #58490 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-27 20:35:09 -07:00
Flora Feng
027b6f3a22
[Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions ( #58939 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 02:01:35 +00:00
Flora Feng
4d07e90653
[Bugfix][Frontend] Sample batched chat completions from the adjusted requests ( #58929 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 01:46:55 +00:00
Ashraf Bhuiyan
b9e2903b89
[Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models ( #58239 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
2026-09-27 21:19:04 -04:00
Thien Tran
39d49539de
[KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel ( #58651 )
2026-09-27 18:18:06 -07:00
Flora Feng
3184226984
[Bugfix][Frontend] Count Responses reasoning tokens per tool round ( #58927 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 00:34:38 +00:00
31afa4f1bb
[ROCm] Bump AITER to v0.1.23 ( #58867 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-27 19:04:58 -05:00
73859fec58
[Frontend] Switch Python Harmony dependency to oss-harmony ( #55128 )
...
Signed-off-by: Anton Peganov <apeganov@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 00:43:56 +04:00
Chao Lei
386ac2573e
[Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeout ( reopens #55763 ) ( #58919 )
2026-09-27 19:47:36 +00:00
Fangzhou Ai
187c81b32d
[ROCm][Bugfix] Fall back to default GEMM for CPU tensors on ROCm builds ( #58923 )
...
Signed-off-by: fai <fangzhouai@gmail.com >
2026-09-27 14:47:07 -05:00
Jiangyun Zhu
231fdb83cc
[Perf][MoE] Use fused MiniMax2 routing with non-unit routed scaling ( #58880 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-09-27 19:37:33 +00:00
Xun Sun and Isotr0py
e7900156e1
[Fast Start] Add /health endpoint for the weight cache daemon ( #58552 )
...
Signed-off-by: Xun Sun <UNIDY2002@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-27 17:48:54 +00:00
2b9b55c7f1
[Bugfix] Support repsonse_format + tool_choice=auto ( #56086 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com >
Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-09-27 21:17:34 +04:00
0376f81530
[Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling in FlashInfer metadata builder ( #57214 )
...
Signed-off-by: frankwang28 <frank.wbb@hotmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-09-27 16:06:38 +00:00
Shanshan Shen
44af287ebe
[ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSeek-V4.1-Flash ( #57407 )
...
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com >
2026-09-27 23:48:01 +08:00
924707f1bf
[CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named jobs ( #57237 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Thang Nguyen <thang.nguyen@inferact.ai >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 05:01:31 -07:00
Nicolò Lucchesi
c8d7a7dd13
[Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA plan under async scheduling ( #58684 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-09-27 13:06:23 +02:00
lesj0610
521235d6df
Merge remote-tracking branch 'origin/main' into sync/lesj/qwen4exp-bugfixes
...
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com >
2026-09-27 18:25:21 +09:00
5567cc1b25
[Mamba] RecoverSSM align: keep a boundary-ending state in its block
...
A speculative window whose accepted tokens end exactly on a recurrent block
boundary selected the following block-table column for the committed state
(commit plan) and recorded it as the running column (postprocess). That
column can still be unallocated, so the next step reads an uninitialized
page. Select the block that holds the last committed token, matching the
(n - 1) // block convention used elsewhere in align mode. Applies to
Kimi-K3 KDA and to the Qwen GDN/PLE RecoverSSM paths that share the plan.
Fix and test from local-inference-lab/vllm a02f585d (Martin Vit, root cause
reported by astra); the GDN/PLE align tests now expect the same convention.
Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com >
Co-authored-by: Martin Vit <martin@voipmonitor.org >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-27 11:12:40 +02:00
24c9772d19
[Attention][CPU] Use zentorch SDPA for CPU MLA prefill ( #54967 )
...
Signed-off-by: Rakul Chauhan <rakul.chauhan@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-27 08:30:55 +00:00
Jürgen Schmied
ed2a9665ea
Merge branch 'main' into gdn-recoverssm
2026-09-27 08:01:32 +02:00
Juntian Liu and Claude Opus 5.5
a9eafde59c
[Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quant on SM100/SM103 ( #58634 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-26 21:16:12 -07:00
MOVZX
5da84df6b7
[BugFix] Honor drop_eagle_block in MambaManager
...
On a hybrid model (full-attention + Mamba/GDN layers, e.g. Qwen3.6-35B-A3B),
enabling MTP/EAGLE speculative decoding together with prefix caching corrupts
the recurrent state, and the server serves wrong answers. #43559 reports
tool-eval accuracy dropping from ~90% to ~50%. Nothing raises an exception
and nothing shows up in the logs, so an operator has no signal that anything
is wrong, and prefix caching then spreads it, because the corrupted state is
hash-reachable and every later request sharing that prefix restores from it.
The root cause is the find_longest_cache_hit method on MambaManager,
at vllm/v1/core/single_type_kv_cache_manager.py:1049 on main, which takes
drop_eagle_block in its signature and never acts on it. Its body searches
right-to-left for the last cached block, builds [null]*i + [cached], and
breaks, referencing neither drop_eagle_block nor .pop(.
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-27 10:45:10 +07:00
Varshith and Claude
fba47397f3
[Bugfix] V1: clear stale allowed_token_ids mask in InputBatch.condense ( #43931 )
...
Signed-off-by: Varshith <kvarshithgowda@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-27 03:44:54 +00:00
MOVZX
5e6656e7a0
[Bugfix][Spec Decode] Add get_top_tokens() to Qwen4ExpMTP draft model
...
use_local_argmax_reduction hard-fails at worker startup when the draft
model lacks get_top_tokens(): the V2 speculator probes for the method
in _validate_local_argmax_reduction and raises, with no fallback to
the full-vocab logits path. Qwen4ExpMTP (nvidia and amd variants) has
compute_logits but no get_top_tokens, so any Qwen4Exp MTP drafter with
the flag enabled crashed both TP ranks during load_model.
Qwen4ExpMTP owns a flat self.lm_head and its compute_logits is exactly
self.logits_processor(self.lm_head, hidden_states), the same shape as
Qwen3_5MTP, so LocalArgmaxMixin applies unchanged: vocab-parallel
argmax with a (value, index) gather, O(2*tp_size) vs O(vocab_size)
communication per draft token. No D2T remapping needed since the
draft vocabulary equals the target vocabulary.
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-27 10:31:47 +07:00
YannikHinteregger and xijiade.aihemaiti
fff03267da
[Bugfix][NIXL] Release a dead peer's NIXL state without waiting for TTL ( #50047 )
...
Signed-off-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com >
Co-authored-by: xijiade.aihemaiti <3146335281@qq.com >
2026-09-26 20:30:09 -07:00