vr8vr8
848dbf89ec
Merge branch 'main' into qwen4exp-qsa-fused-draft-metadata
2026-09-23 23:37:13 +03:00
陈志谦 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
97b1b12117
[Docs] Fix docstring typos (output_dytpe, kwrags, Abbrivations) ( #55936 )
...
Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 16:36:57 -04:00
Divy and Claude Opus 5
e26547adfb
[Bugfix] Pick a KV block size supported by every attention backend ( #49845 )
...
Signed-off-by: Divy <divy@coralbricks.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-23 16:36:35 -04:00
Thang Nguyen and Claude Opus 5.5
34b069080d
[CI] Select one GPU for the H200 initialized snapshot E2E step ( #58351 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-23 13:08:51 -07:00
Lio Einaudi
711fc55c10
[Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M ( #57586 )
...
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com >
2026-09-23 14:39:38 -04:00
bilikas and Claude Opus 5.5
31d78a21b9
[Spec Decode][Qwen3.8] Enable fused draft metadata updates for the QSA side cache
...
QSAMetadataBuilder did not set supports_draft_decode_metadata_update, so
MTP with num_speculative_tokens > 1 fell back to rebuilding attention
metadata between draft steps for the whole model.
Opt the builder in. update_draft_decode_metadata re-launches the builder's
own _build_qsa_metadata_kernel in place on the metadata's persistent
tensors, so draft-step token_to_req, logical_positions, visible_blocks,
slot_mapping and k_work_metadata come from the same code path as build().
The kernel launch is factored into _launch_qsa_metadata_kernel, and
QSAForwardMetadata carries common_slot_mapping and num_mapped_tokens so
the update does not read query_start_loc_cpu. The PyTorch fallback builder
keeps the rebuild path.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Signed-off-by: bilikas <valdas@vbtech.eu >
2026-09-23 20:58:12 +03:00
CBP and Misha Goin
f9ad9dd6b4
Doc: add DiffusionGemma to supported models ( #46466 )
...
Signed-off-by: Bruce <Bruce798858117@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-23 17:14:30 +00:00
Simon Danielsson
b5ea0c7107
[Bugfix][ROCm] Fix startup OOM in AITER MLA FP8 prefill workspace sizing ( #57923 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-23 11:44:12 -05:00
Dao007forever and Codex
c4f6ce4874
[Bugfix][NIXL] Restore successful push completion reporting ( #58188 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 09:39:04 -07:00
c4cfd7007b
[CI][ROCm] Add an MI355 Kimi-K3 unit test group ( #58012 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 11:25:51 -05:00
Andreas Karatzas and OpenAI Codex
8033bd08c1
[ROCm][CI] Validate Mooncake and NIXL prefill/decode accuracy ( #58095 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 11:16:31 -05:00
b715699df4
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE ( #50212 )
...
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com >
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 11:11:57 -05:00
fxmarty-amd and mgoin
0549e8d0ab
[Quark] Remove quark-specific silent online quantization ( #51800 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-23 11:35:30 -04:00
f69cc7c178
[Quantization][Bugfix] Bump humming-kernels to 0.1.16 ( #58054 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-23 08:22:45 -07:00
HDCharles
9f07d023d0
[Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors ( #46528 )
2026-09-23 10:23:11 -04:00
Ankit-Jaiswal-AMD and Douglas Lehr
88afb9dcac
[CPU] Adds support for fp32 attention sinks ( #56252 )
...
Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 08:55:09 -05:00
Shuolei Wang
9ed81581a8
[Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… ( #58051 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
2026-09-23 21:03:21 +08:00
d95a896ea6
[Feature][Frontend] Add DeepSeek-V4 FIM completion rendering ( #44229 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 13:02:00 +00:00
157bcb7c48
[DSpark] Support pipeline-parallel targets in aggregated serving ( #56956 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-23 04:52:28 -07:00
15ed1262e7
[Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine ( #49648 )
...
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 10:37:40 +00:00
Tianyu Guo
c961121519
[Bugfix] Disable prefix caching for encoder-only before model config hooks ( #58287 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-23 09:25:32 +00:00
Andreas Karatzas
a22c637ed9
[ROCm][Test] Check GDN prefill numerics and output ownership ( #58091 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 02:19:58 -07:00
Yan Ma and Kunshang Ji
e34489a543
[XPU] upgrade to PyTorch 2.14 ( #56013 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-23 09:18:52 +00:00
Gabriel Wu and Kimi Code
c843f0cab7
[Bugfix][SM120][MLA] Support NoPE sparse MLA (GLM-5.3-Flash) on the FlashInfer SM120 backend ( #55277 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-23 02:03:20 -07:00
Nick Hill
afec265bca
[MRV2] Miscellaneous code cleanup ( #57980 )
2026-09-23 01:51:52 -07:00
Andreas Karatzas and OpenAI Codex
36bc694889
[ROCm][CI] Include Python tooling in ROCm CI artifacts ( #58271 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 16:44:54 +08:00
Andreas Karatzas
56b3acb55b
[ROCm][Bugfix] Keep zero MiniMax MXFP8 activation blocks finite ( #58089 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 16:42:33 +08:00
1904ed9402
[Perf][ROCm][Attention] Narrow the Triton prefill-attention KV tile on RDNA3/RDNA4 ( #58225 )
...
Signed-off-by: Jipeng Li <jipengli@amd.com >
Co-authored-by: GitHub Copilot CLI <noreply@github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 08:39:27 +00:00
vllm-agent and kimi-no-na-wa
955bd6abef
[CI][Bugfix] Extend groupwise rms_norm scale tolerance to CUDA ( #58252 )
...
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
2026-09-23 01:29:11 -07:00
Jiangyun Zhu and OpenAI Codex
6632ed559e
[Perf][Attention] Avoid CPU-GPU sync in DCP sequence lengths ( #58169 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 16:28:43 +08:00
Andreas Karatzas and OpenAI Codex
96e203b08e
[ROCm][Bugfix] Register MRV2 sampler JIT warmups ( #58092 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 03:17:13 -05:00
Andreas Karatzas and OpenAI Codex
26879f3260
[ROCm][Compile] Fuse AITER static FP8 attention output ( #58099 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 03:16:52 -05:00
aoshen02 and Yongye Zhu
973a3be780
[Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions ( #57732 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-23 01:16:20 -07:00
Nicolò Lucchesi and Yifan Qiao
9488318c6a
[Mooncake] Address review nits from #56855 ( #57174 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Yifan Qiao <17067717+ivanium@users.noreply.github.com >
2026-09-23 10:15:45 +02:00
Andreas Karatzas and OpenAI Codex
153242a314
[ROCm][CI] Add MI355 dense NVFP4 and MoRI kernel mirrors ( #58281 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 03:06:15 -05:00
Andreas Karatzas and OpenAI Codex
a9f07d0bc5
[ROCm][CI] Mirror the three TurboQuant evaluation groups on MI355 ( #58282 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 02:37:23 -05:00
9127170b88
[Quantization] Select per-token NVFP4 MoE backends explicitly ( #57176 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 07:30:58 +00:00
Andreas Karatzas and OpenAI Codex
0f2a15c927
[Tests] Select V2 for diffusion scheduler unit tests ( #58272 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 02:04:22 -05:00
00CC
eb2e91d3ba
[ROCm][Perf] Use wvSplitK for single-output GEMMs ( #53283 )
...
Signed-off-by: tangzzycc <3081129260@qq.com >
2026-09-23 06:38:17 +00:00
Ting SUN
b33c8a657d
[Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls ( #46303 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-09-23 14:31:10 +08:00
Jee Jee Li
de6674b002
[MoE] Use GateLinear for all MoE models ( #58234 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-23 05:40:13 +00:00
xiangdong
e975732152
[XPU][CI] Deselect tests/v1/spec_decode/test_mtp.py::test_glm_mtp_defers_lm_head ( #58237 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
2026-09-23 05:20:51 +00:00
88aa0d287d
[Spec decode] Support variable-length decode for Kimi-K3 adaptive ver ( #52988 )
...
Signed-off-by: Albert Cheng <albecheng@nvidia.com >
Signed-off-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-bia01.bia.clusters.nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-09-22 20:43:32 -07:00
Wentao Ye
355d789f0d
[Compilation] Fix QuTLASS compilation with PyTorch 2.13 ( #58173 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-22 20:43:28 -07:00
Michał Ganczarenko and Claude
73c59d365b
[XPU] Wire up SYCL apply_rotary_emb kernel in ApplyRotaryEmb ( #55721 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Signed-off-by: Michał Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-22 20:43:24 -07:00
94f4170df3
[Bugfix] Set worker runtime threads before profiling and compilation ( #55891 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-22 21:12:20 -05:00
Andreas Karatzas and OpenAI Codex
e581e14002
[ROCm][CI] Stage G gating ( #50922 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-22 19:04:54 -07:00
8d16ca6cc2
[CI][Bugfix] Update IPC test caller for #57312 's _apply_entries signature ( #58107 )
...
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-22 18:50:06 -07:00
Andreas Karatzas
0eb42dbbfc
[ROCm][Compile] Support BF16 AsyncTP fusion ( #58098 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-22 20:36:38 -05:00
Fangzhou Ai
f9dce295c9
[ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce ( #57451 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
2026-09-23 01:23:55 +00:00