Andreas Karatzas and OpenAI Codex
23110a0c9e
[Bugfix] Capture prefill kernels for mixed FULL graphs ( #58275 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-24 02:05:07 -07:00
jackLei and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
bbc4ddeca9
[Bugfix][V1] Reject encoder-cache hits with mismatched embedding counts ( #57696 )
...
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-24 08:40:47 +00:00
Li, Jiang
b44895cf93
[CPU] Use pre-built triton ( #58140 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-24 16:12:34 +08:00
Talor Abramovich
28282ffe34
[Feature][Frontend] Request JSON body debug logging on --enable-log-requests flag ( #58163 )
...
Signed-off-by: talora <talora@nvidia.com >
2026-09-24 08:08:40 +00:00
Thien Tran and OpenAI Codex
00b7847c80
[Perf] Use Conv3dLayer for MiniMax M3 patch embedding ( #58512 )
...
Signed-off-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-24 00:41:36 -07:00
1291bdb85e
[EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only ( #58086 )
...
Signed-off-by: grYe99 <guorongye99@gmail.com >
Co-authored-by: grYe99 <guorongye99@gmail.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-09-24 00:41:15 -07:00
Andreas Karatzas and OpenAI Codex
c3f5270270
[ROCm][CI] Add the MI355 TurboQuant t3nc mirror ( #58432 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-24 02:16:09 -05:00
67ddac3ea8
[CPU][GDN] Support NIXL DS convolution-state layout ( #53300 )
...
Signed-off-by: Li, Tianmu <tianmu.li@intel.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-24 14:55:20 +08:00
Andreas Karatzas and Codex
153f2ba1b9
[Compile][CI] Honor Triton cache overrides and add AMD timeout headroom ( #58474 )
...
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-24 01:27:07 -05:00
e028554a1f
[Rust Frontend] Support --sse-keep-alive-interval ( #58306 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 06:19:14 +00:00
Biswa Panda and Codex
cf78797cb7
[Rust Frontend] Pass vision preprocessing context for Nemotron-H ( #57634 )
...
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.
Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.
Co-authored-by: Codex <noreply@openai.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 14:08:26 +08:00
lucamotz and Claude Opus 5.5
ee2f3cdf12
[Bugfix][Spec Decode] Separate DSpark width from MTP stage validation ( #54631 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-24 06:08:22 +00:00
44bf011f44
[Dependency] Upgrade FlashInfer version to 0.7.0 ( #58069 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-23 22:45:23 -07:00
Bugen Zhao and Claude Opus 5.5
a1b6763df6
[Rust Frontend] Accept custom chat roles for HF templates ( #58311 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 13:43:34 +08:00
Bugen Zhao and Claude Opus 5.5
a5603f21ae
[Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op ( #58330 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 13:43:24 +08:00
8b06a343ec
[Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 ( #56168 )
...
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-24 13:15:17 +08:00
aoshen02
7f1a5398e9
Fix full logprobs in token-in/token-out responses ( #58488 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-09-24 04:46:16 +00:00
Kapil Arya and Or Ozeri
70fc359d25
[Bugfix][KV Offload] Retain offload event metadata through batch translation ( #57453 )
...
Signed-off-by: Kapil Arya <kapila@nvidia.com >
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-09-24 07:09:12 +03:00
Jhao-Ting Chen
4fb767ff0a
[Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection ( #53175 )
...
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
2026-09-23 20:29:06 -07:00
e78e367c6e
[ROCm] Give turboquant boundary layers a layout-compatible backend ( #54988 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 22:00:07 -05:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Robert Shaw
6697a7dd30
[Scheduler] Tune --long-prefill-token-threshold adaptiveness ( #58459 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
2026-09-23 22:23:18 -04:00
e9f167129c
[CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs ( #58369 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 21:01:58 -05:00
Bob Ok and Wentao Ye
bcdacfc1ff
[Test][Determinism] Cover chunked prefill in the batch-invariance suite ( #55612 )
...
Signed-off-by: Bob Ok <49168652+blipbyte@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-23 21:21:56 -04:00
Juntian Liu and Claude Opus 5.5
9ef37771be
[Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap ( #58215 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 00:54:27 +00:00
Aarushi Jain and Andreas Karatzas
e0f03eb4f4
[CI] Share BF16 baselines across quantization comparison tests ( #58469 )
...
Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 19:26:09 -05:00
Kevin H. Luu and Claude Opus 5.5
7ef0d69ea0
[5/12][ci-selector][CI] Skip the Proton GPU test when another CUPTI tool is injected ( #58455 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 17:25:41 -07:00
Nick Hill
310f15d354
[Bugfix][MRV2] Align dummy idx_mapping dtype to avoid runtime jit ( #58462 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-23 16:54:48 -07:00
Misha Goin
89d9fd83e2
[CI] Disable JIT warmup by default in VllmRunner ( #58452 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-23 18:53:58 -05:00
Kevin H. Luu and Claude Opus 5.5
21ee743f47
[CI][Bugfix] Limit MRV2 sampler JIT warmup registration to ROCm ( #58465 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 16:11:50 -07:00
djramic and Andreas Karatzas
5fcc6e7c73
[ROCm][Bugfix] Fix TileLang mHC fused RMSNorm on 64-wide wavefronts ( #58419 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 23:03:45 +00:00
Andreas Karatzas
8b660ce96b
[ROCm][Test] Cover MoRI graph replay and output lifetime ( #58093 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 14:16:51 -07:00
陈志谦 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
97b1b12117
[Docs] Fix docstring typos (output_dytpe, kwrags, Abbrivations) ( #55936 )
...
Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 16:36:57 -04:00
Divy and Claude Opus 5
e26547adfb
[Bugfix] Pick a KV block size supported by every attention backend ( #49845 )
...
Signed-off-by: Divy <divy@coralbricks.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-23 16:36:35 -04:00
Thang Nguyen and Claude Opus 5.5
34b069080d
[CI] Select one GPU for the H200 initialized snapshot E2E step ( #58351 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-23 13:08:51 -07:00
Lio Einaudi
711fc55c10
[Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M ( #57586 )
...
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com >
2026-09-23 14:39:38 -04:00
CBP and Misha Goin
f9ad9dd6b4
Doc: add DiffusionGemma to supported models ( #46466 )
...
Signed-off-by: Bruce <Bruce798858117@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-23 17:14:30 +00:00
Simon Danielsson
b5ea0c7107
[Bugfix][ROCm] Fix startup OOM in AITER MLA FP8 prefill workspace sizing ( #57923 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-23 11:44:12 -05:00
Dao007forever and Codex
c4f6ce4874
[Bugfix][NIXL] Restore successful push completion reporting ( #58188 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 09:39:04 -07:00
c4cfd7007b
[CI][ROCm] Add an MI355 Kimi-K3 unit test group ( #58012 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 11:25:51 -05:00
Andreas Karatzas and OpenAI Codex
8033bd08c1
[ROCm][CI] Validate Mooncake and NIXL prefill/decode accuracy ( #58095 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 11:16:31 -05:00
b715699df4
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE ( #50212 )
...
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com >
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 11:11:57 -05:00
fxmarty-amd and mgoin
0549e8d0ab
[Quark] Remove quark-specific silent online quantization ( #51800 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-23 11:35:30 -04:00
f69cc7c178
[Quantization][Bugfix] Bump humming-kernels to 0.1.16 ( #58054 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-23 08:22:45 -07:00
HDCharles
9f07d023d0
[Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors ( #46528 )
2026-09-23 10:23:11 -04:00
Ankit-Jaiswal-AMD and Douglas Lehr
88afb9dcac
[CPU] Adds support for fp32 attention sinks ( #56252 )
...
Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 08:55:09 -05:00
Shuolei Wang
9ed81581a8
[Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… ( #58051 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
2026-09-23 21:03:21 +08:00
d95a896ea6
[Feature][Frontend] Add DeepSeek-V4 FIM completion rendering ( #44229 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 13:02:00 +00:00
157bcb7c48
[DSpark] Support pipeline-parallel targets in aggregated serving ( #56956 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-23 04:52:28 -07:00
15ed1262e7
[Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine ( #49648 )
...
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 10:37:40 +00:00