Gabriel Wu and Claude Opus 5.5
dcd07967ca
[CPU] Keep the hybrid block size across the engine core's config rerun ( #61805 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 12:06:03 +08:00
ce49c47cbb
[Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel ( #62046 )
...
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com >
Co-authored-by: Bowen Wang <bowen@inferact.ai >
2026-10-17 02:26:57 +00:00
Sherif Waly and vllm-agent
ce8853d27a
[Bugfix][KV Offload] Allow explicit SimpleCPUOffload block counts ( #61674 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 02:11:12 +00:00
71c20f73f2
[FlashInfer] Migrate autotune warmup to flashinfer.autotune_v2 managed cache ( #55671 )
...
Signed-off-by: Yang Xu <yanxu@nvidia.com >
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Signed-off-by: YangXu <yanxu@nvidia.com >
Co-authored-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-17 02:08:28 +00:00
Shuolei Wang and aoshen02
b3a7e2ff6e
[RL][Frontend] Return weight-version spans on OpenAI completions and chat ( #61347 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-17 02:08:07 +00:00
6b6a6c5c98
[RL] Support per-parameter source layouts for NCCL M2N ( #60934 )
...
Signed-off-by: pkousha <pkousha@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-16 21:41:17 -04:00
Harry Mellor and Claude Opus 5.5
f8112704d5
[Chore] Use upstream Transformers InklingProcessor ( #61883 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 01:04:54 +00:00
f019e3e105
[ROCm] attn res + sigmoid_mul + conv fusions ( #52968 )
...
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-17 00:54:54 +00:00
Vadim Gimpelson and Claude
783380af39
[Bugfix] Fix NaN output of the Mamba2 chunk scan kernel on sm90 ( #61718 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-16 17:24:51 -07:00
Wei Zhao and Codex
be25977532
[CI][Tests] Account for FP8 KV-cache rounding in MLA reference ( #62067 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-16 17:10:08 -07:00
523d74e2ac
[Bugfix][LoRA] Fix garbage/NaN lora_expand output on SM90 with fp32 i… ( #61906 )
...
Signed-off-by: BeichenHuang <polarishuang0.0@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-16 19:42:30 -04:00
a94c315711
[DP] Engines own wave wake-ups; drop the front-end FIRST_REQ ( #59559 )
...
Signed-off-by: fh3 <a@b>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 19:41:54 -04:00
b544587408
[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning ( #53558 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-16 23:16:47 +00:00
alp and mottopanikeiku
2268d83bd9
[Bugfix][Multimodal] Honor RGB PNG transparency when loading images ( #61986 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-16 23:09:12 +00:00
28e92eeff2
[ROCm][Perf][Kimi-K3] Use AITER Gluon chunk_kda for KDA prefill ( #59552 )
...
Signed-off-by: Omar Muhammad <omar.muhammad@amd.com >
Signed-off-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Nicholas Susanto <nicholas.susanto@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 22:58:56 +00:00
Afonso Ribeiro and Flora Feng
b088eba944
[Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) ( #62035 )
...
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-16 22:46:02 +00:00
596e530675
[Kernel] Add router GEMM kernels for (5120, 256) and BF16 output to ll_bf16_gemm ( #62042 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Lain <fusiyuan2000@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 15:39:46 -07:00
153a17166e
[Frontend] Add the Beam reasoning and tool parsers ( #62052 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 14:08:29 -07:00
Dan Blanaru and Claude Opus 5.5
9731f30a03
[Bugfix] Keep VLLM_RAY_BUNDLE_INDICES out of the compile cache key ( #61211 )
...
Signed-off-by: DanBlanaru <48605845+DanBlanaru@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 20:59:48 +00:00
68ebea789a
[Perf][MoE] Skip padded work in SM100 FP4 DeepGEMM experts ( #59044 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 16:20:52 -04:00
cc2e4fecf2
[ROCm] [fusion] support ROCm aiter fusion of per tensor quant + rmsnorm ( #52831 )
...
Signed-off-by: Andy Friedrich <afriedri@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 15:19:07 -05:00
9eb7fb9157
[Build] Remove unused tools/install_deepgemm.sh ( #61265 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 15:57:49 -04:00
cjackal
663e535640
[Perf][Spec Decode] Skip bonus-logit reduction for short drafts in rejection sampling ( #61451 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-10-16 12:42:48 -07:00
Oxana Korzh and Claude Opus 5.5
5bf3d9f665
[ROCm][Kimi-K3] Support DSpark adaptive verification on MI355X ( #59824 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-16 13:53:49 -05:00
djramic
307c5844ad
[ROCm][CI] Enable batch invariance for the ngram scheduling accuracy test ( #62048 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-16 18:36:14 +00:00
Guan-Ming Chiu and vllm-agent
3dc6def360
[Attention][Spec Decode] Enable varlen Mamba2 decode for adaptive verification ( #53036 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 11:23:59 -07:00
djramic
e7a5bdb454
[ROCm][CI] Move Spec Decode Speculators + MTP to MI355 ( #62053 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
2026-10-16 18:22:38 +00:00
alberto
ae8f41238c
[Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts ( #61857 )
...
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com >
2026-10-16 12:54:42 -04:00
cffc0cfa31
[CI] Fix B200 FusedMoE Layer Kernels hang on NVSHMEM IB setup ( #61695 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 12:00:12 -04:00
Andreas Karatzas and Claude
706791d516
[CI][AMD] Move 17 test groups from MI300 to MI355 and test large DeepSeek/Nemotron evals on AITER ( #61955 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-16 10:58:59 -05:00
Sawyer Bowerman and Codex
76081b9ee4
[Bugfix][Model] Honor quantization config in Inkling MTP ( #61921 )
...
Signed-off-by: Sawyer Bowerman <sbowerma@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-16 15:49:26 +00:00
bnellnm
1a5b187066
Add @bnellnm to CODEOWNERS ( #62025 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-10-16 11:16:03 -04:00
Wentao Ye
8e25bbca19
[Refactor] Remove dead code multiple places ( #61843 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-10-16 15:01:07 +00:00
8a6c0449b3
[Bugfix][ROCm][GLM-5.2] Stop forcing skip_final_all_reduce on AMD DSA model path ( #58589 )
...
Signed-off-by: avininjamay8 <Avinash.Paul@amd.com >
Signed-off-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: avininjamay8 <avpaul@amd.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 21:38:55 +08:00
Ayushman Singh
de7eeaa87b
[Perf] Share Triton JIT warmup compilation pool across registered kernels ( #61422 )
2026-10-16 08:59:42 -04:00
MOVZX
be95edb7f6
Merge branch 'main' into patches
...
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 17:35:42 +07:00
Martin Hickey
64175d3827
[Bugfix][Frontend] Keep the parser's tool call IDs in batch derender ( #61852 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-16 10:27:17 +00:00
alp and mottopanikeiku
4ab99ff136
[Bugfix][Frontend] Honor the constructor chat template in offline scoring ( #61990 )
...
Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com >
2026-10-16 10:20:46 +00:00
024c5188bf
[Frontend] Resolve logprobs on the parsed streaming derender path ( #60437 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-16 18:04:15 +08:00
Hexiang Wang
9a16d43ef0
[Bugfix][Mooncake] Fix MultiConnector stats and proxy SSE responses ( #61218 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
Signed-off-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com >
2026-10-16 17:54:48 +08:00
MOVZX and Hinata
8dc54f35dd
[Offloader] Stream every multimodal tower parameter to host RAM
...
Qwen3_VisionTransformer derived its compute device from
patch_embed.proj.weight and fed pos_embed.weight straight into the
position-embedding interpolation kernel, so the last two anchor weights
had to stay in VRAM for a tower to be streamable at all.
Record the device the tower was built on before the offloader moves
anything, and fall back to it when no weight is on the accelerator.
Move the interpolation table to that device per encode, which costs one
8.4 MiB host-to-device copy for a 456 MiB tower. Towers now leave no
parameters in VRAM between encodes; only small buffers such as the fused
input norm and rotary inv_freq remain.
Also let a tower restrict streaming through tower_offload_children, and
warn when a tower resolves its device property to another device type,
so a tower that reads an anchor weight reports the cause instead of a
Triton pointer error.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 16:07:53 +07:00
zofia
07f3857fb9
[XPU][Linear][MXFP4] enable torch as mxfp4 linear backend on xpu ( #51965 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com >
2026-10-16 17:07:05 +08:00
MOVZX and Hinata
18c06ed406
[Offloader] Stream multimodal towers from host RAM with bulk-copy
...
Add --offload-towers: place multimodal tower weights in pinned host
memory and bulk-copy each unit to the GPU only during its forward,
keeping towers out of VRAM between encodes. Unlike UVA zero-copy,
whole-tensor H2D transfers avoid re-reading operand tiles over PCIe
inside the GEMM inner loop. Tower streaming is independent of layer
group scheduling, so group_size may be 0 for tower-only offload.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-16 14:32:14 +07:00
fa75d3456d
[Feature][PCP] Add DeepSeek-V4.1-Flash support ( #59857 )
...
Signed-off-by: wangyicong <wangyicong@bytedance.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-15 23:17:51 -07:00
Chaojun Zhang and vllm-agent
074c83a09a
[CI][Intel] Skip ROCm-only test_topk_gating_enabled_by_cdna on XPU ( #61932 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: vllm-agent <claw@inferact.ai >
2026-10-16 13:44:32 +08:00
Wei Zhao
e365792997
[Dependency] Upgrade TokenSpeed MLA to 2.0.3 ( #61689 )
...
Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-15 22:28:59 -07:00
dskwe
49cb2bbf71
[Bugfix] Honor code_revision in config-based tokenizer loading ( #61611 )
...
Signed-off-by: Yuan Li <dskwelmcy@163.com >
2026-10-16 05:21:12 +00:00
4ac1f8d2ac
[Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends ( #61574 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 12:53:41 +08:00
maithilijoshi20 and Cyrus Leung
92dc3dd393
[Bugfix] Honor OPT token embedding width for prompt embeddings ( #61736 )
...
Signed-off-by: maithilijoshi20 <maithilij2003@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-16 04:41:50 +00:00
Andreas Karatzas and Claude
b466fbacd2
[ROCm][Perf] Any-shape low-latency MoE router GEMM on MI355 ( #61895 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-15 23:22:31 -05:00