bilikas and Claude Opus 5.5
31d78a21b9
[Spec Decode][Qwen3.8] Enable fused draft metadata updates for the QSA side cache
...
QSAMetadataBuilder did not set supports_draft_decode_metadata_update, so
MTP with num_speculative_tokens > 1 fell back to rebuilding attention
metadata between draft steps for the whole model.
Opt the builder in. update_draft_decode_metadata re-launches the builder's
own _build_qsa_metadata_kernel in place on the metadata's persistent
tensors, so draft-step token_to_req, logical_positions, visible_blocks,
slot_mapping and k_work_metadata come from the same code path as build().
The kernel launch is factored into _launch_qsa_metadata_kernel, and
QSAForwardMetadata carries common_slot_mapping and num_mapped_tokens so
the update does not read query_start_loc_cpu. The PyTorch fallback builder
keeps the rebuild path.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Signed-off-by: bilikas <valdas@vbtech.eu >
2026-09-23 20:58:12 +03:00
15ed1262e7
[Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine ( #49648 )
...
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 10:37:40 +00:00
Tianyu Guo
c961121519
[Bugfix] Disable prefix caching for encoder-only before model config hooks ( #58287 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-23 09:25:32 +00:00
Andreas Karatzas
a22c637ed9
[ROCm][Test] Check GDN prefill numerics and output ownership ( #58091 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 02:19:58 -07:00
Yan Ma and Kunshang Ji
e34489a543
[XPU] upgrade to PyTorch 2.14 ( #56013 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-23 09:18:52 +00:00
Gabriel Wu and Kimi Code
c843f0cab7
[Bugfix][SM120][MLA] Support NoPE sparse MLA (GLM-5.3-Flash) on the FlashInfer SM120 backend ( #55277 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-23 02:03:20 -07:00
Nick Hill
afec265bca
[MRV2] Miscellaneous code cleanup ( #57980 )
2026-09-23 01:51:52 -07:00
Andreas Karatzas and OpenAI Codex
36bc694889
[ROCm][CI] Include Python tooling in ROCm CI artifacts ( #58271 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 16:44:54 +08:00
Andreas Karatzas
56b3acb55b
[ROCm][Bugfix] Keep zero MiniMax MXFP8 activation blocks finite ( #58089 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 16:42:33 +08:00
1904ed9402
[Perf][ROCm][Attention] Narrow the Triton prefill-attention KV tile on RDNA3/RDNA4 ( #58225 )
...
Signed-off-by: Jipeng Li <jipengli@amd.com >
Co-authored-by: GitHub Copilot CLI <noreply@github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 08:39:27 +00:00
vllm-agent and kimi-no-na-wa
955bd6abef
[CI][Bugfix] Extend groupwise rms_norm scale tolerance to CUDA ( #58252 )
...
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
2026-09-23 01:29:11 -07:00
Jiangyun Zhu and OpenAI Codex
6632ed559e
[Perf][Attention] Avoid CPU-GPU sync in DCP sequence lengths ( #58169 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 16:28:43 +08:00
Andreas Karatzas and OpenAI Codex
96e203b08e
[ROCm][Bugfix] Register MRV2 sampler JIT warmups ( #58092 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 03:17:13 -05:00
Andreas Karatzas and OpenAI Codex
26879f3260
[ROCm][Compile] Fuse AITER static FP8 attention output ( #58099 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 03:16:52 -05:00
aoshen02 and Yongye Zhu
973a3be780
[Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions ( #57732 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-23 01:16:20 -07:00
Nicolò Lucchesi and Yifan Qiao
9488318c6a
[Mooncake] Address review nits from #56855 ( #57174 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Yifan Qiao <17067717+ivanium@users.noreply.github.com >
2026-09-23 10:15:45 +02:00
Andreas Karatzas and OpenAI Codex
153242a314
[ROCm][CI] Add MI355 dense NVFP4 and MoRI kernel mirrors ( #58281 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 03:06:15 -05:00
Andreas Karatzas and OpenAI Codex
a9f07d0bc5
[ROCm][CI] Mirror the three TurboQuant evaluation groups on MI355 ( #58282 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-23 02:37:23 -05:00
9127170b88
[Quantization] Select per-token NVFP4 MoE backends explicitly ( #57176 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 07:30:58 +00:00
Andreas Karatzas and OpenAI Codex
0f2a15c927
[Tests] Select V2 for diffusion scheduler unit tests ( #58272 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 02:04:22 -05:00
00CC
eb2e91d3ba
[ROCm][Perf] Use wvSplitK for single-output GEMMs ( #53283 )
...
Signed-off-by: tangzzycc <3081129260@qq.com >
2026-09-23 06:38:17 +00:00
Ting SUN
b33c8a657d
[Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls ( #46303 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-09-23 14:31:10 +08:00
Jee Jee Li
de6674b002
[MoE] Use GateLinear for all MoE models ( #58234 )
...
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-09-23 05:40:13 +00:00
xiangdong
e975732152
[XPU][CI] Deselect tests/v1/spec_decode/test_mtp.py::test_glm_mtp_defers_lm_head ( #58237 )
...
Signed-off-by: zengxian <xiangdong.zeng@intel.com >
2026-09-23 05:20:51 +00:00
88aa0d287d
[Spec decode] Support variable-length decode for Kimi-K3 adaptive ver ( #52988 )
...
Signed-off-by: Albert Cheng <albecheng@nvidia.com >
Signed-off-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-bia01.bia.clusters.nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-09-22 20:43:32 -07:00
Wentao Ye
355d789f0d
[Compilation] Fix QuTLASS compilation with PyTorch 2.13 ( #58173 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-22 20:43:28 -07:00
Michał Ganczarenko and Claude
73c59d365b
[XPU] Wire up SYCL apply_rotary_emb kernel in ApplyRotaryEmb ( #55721 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
Signed-off-by: Michał Ganczarenko <michal.ganczarenko@intel.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-22 20:43:24 -07:00
94f4170df3
[Bugfix] Set worker runtime threads before profiling and compilation ( #55891 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-22 21:12:20 -05:00
Andreas Karatzas and OpenAI Codex
e581e14002
[ROCm][CI] Stage G gating ( #50922 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-22 19:04:54 -07:00
8d16ca6cc2
[CI][Bugfix] Update IPC test caller for #57312 's _apply_entries signature ( #58107 )
...
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-22 18:50:06 -07:00
Andreas Karatzas
0eb42dbbfc
[ROCm][Compile] Support BF16 AsyncTP fusion ( #58098 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-22 20:36:38 -05:00
Fangzhou Ai
f9dce295c9
[ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce ( #57451 )
...
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com >
2026-09-23 01:23:55 +00:00
Tianyu Guo and Cyrus Leung
3ee2906f47
[EPD] Support metadata-only audio inputs ( #57887 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-23 09:03:50 +08:00
826e300cde
[Bugfix] Skip VllmConfig re-validation for with_hf_config submodel views ( #58212 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.ai >
Co-authored-by: Roger Wang <rogerw@inferact.ai >
2026-09-23 00:50:17 +00:00
d110c2f19c
[ROCm] Use silu_and_mul_with_clamp's torch._C op ( #52052 )
...
Signed-off-by: Tres Popp <tres.popp@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-22 18:30:14 -05:00
1c0eee919d
[ROCm][Model][Bugfix] Enable GLM-5.2-MXFP4 on the deepseek_v32 path and fix sparse attention correctness ( #51915 )
...
Signed-off-by: Jack Hu <Jack.Hu@amd.com >
Signed-off-by: Jack Hu <jack.hu@amd.com >
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com >
Co-authored-by: James E T Smith <jamesETsmith@users.noreply.github.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com >
2026-09-22 18:08:51 -05:00
stefankoncarevic and Andreas Karatzas
9646f53064
[Bugfix][ROCm] Use the platform FP8 range in the concat MLA q test ( #58153 )
...
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-22 18:05:33 -05:00
rasmith
111d7d7be2
[ROCm][Build][The Rock] Bump Triton version to 3.8.x tip-of-tree with source build in The Rock image ( #58006 )
...
Signed-off-by: Randall Smith <Randall.Smith@amd.com >
2026-09-22 18:05:13 -05:00
Michael Lapshin
d421a2e8fc
[Perf] Remove CPU-GPU sync in heterogeneous vocabulary speculative decoding ( #57396 )
2026-09-22 18:34:19 -04:00
Thang Nguyen and Claude Opus 5.5
4edb55169f
[CI] Shard (H200 MIG 18GB) Spec Decode Draft Model across whole-directory replicas ( #58193 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-22 14:50:18 -07:00
Thang Nguyen and Kimi
4ec0b95c44
[CI] Split LM Eval TurboQuant KV Cache into per-config jobs ( #57113 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Kimi <noreply@moonshot.ai >
2026-09-22 14:49:06 -07:00
Misha Goin
5c3b61e9c4
[Core] Disable JIT warmup in eager mode ( #58197 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-22 17:35:23 -04:00
Misha Goin
01e15bc94c
[Bugfix] Backport Inductor custom-op pattern matching fix ( #58189 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-22 14:26:49 -07:00
Andrey Talman
ecdd11d0e0
[CI] Build the torch-nightly image on Ubuntu 24.04 ( #58204 )
2026-09-22 14:15:57 -07:00
Misha Goin
25965ee583
[Kernel] Remove AllSpark INT8 W8A16 GEMM backend ( #58001 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-22 14:07:34 -07:00
1b3b88ec2b
[Core] structured generation mode for DiffusionGemma model (Jev-like) ( #57250 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Razorback16 <razorback16@protonmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Razorback16 <razorback16@protonmail.com >
2026-09-22 20:51:17 +00:00
Wentao Ye
fae5cbfc8b
[Refactor] Remove dead code multiple places ( #58002 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-22 16:36:03 -04:00
lucamotz
276db94fb2
[Bugfix][Model][Spec Decode] Defer disposable GLM MTP head ( #55442 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
2026-09-22 20:08:27 +00:00
hcl and mgoin
71a2392845
[Bugfix][V1] Read ModelState max_model_len from model config ( #58149 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-22 19:45:04 +00:00
Divakar Verma and Andreas Karatzas
d1f63f2af9
[ROCm][CI] Add GELU activation for AiterExperts in the modular-kernel coverage ( #58030 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-22 14:39:13 -05:00