e028554a1f
[Rust Frontend] Support --sse-keep-alive-interval ( #58306 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 06:19:14 +00:00
Biswa Panda and Codex
cf78797cb7
[Rust Frontend] Pass vision preprocessing context for Nemotron-H ( #57634 )
...
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.
Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.
Co-authored-by: Codex <noreply@openai.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 14:08:26 +08:00
lucamotz and Claude Opus 5.5
ee2f3cdf12
[Bugfix][Spec Decode] Separate DSpark width from MTP stage validation ( #54631 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-24 06:08:22 +00:00
44bf011f44
[Dependency] Upgrade FlashInfer version to 0.7.0 ( #58069 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-23 22:45:23 -07:00
Bugen Zhao and Claude Opus 5.5
a1b6763df6
[Rust Frontend] Accept custom chat roles for HF templates ( #58311 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 13:43:34 +08:00
Bugen Zhao and Claude Opus 5.5
a5603f21ae
[Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op ( #58330 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-24 13:43:24 +08:00
8b06a343ec
[Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 ( #56168 )
...
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-24 13:15:17 +08:00
aoshen02
7f1a5398e9
Fix full logprobs in token-in/token-out responses ( #58488 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
2026-09-24 04:46:16 +00:00
Kapil Arya and Or Ozeri
70fc359d25
[Bugfix][KV Offload] Retain offload event metadata through batch translation ( #57453 )
...
Signed-off-by: Kapil Arya <kapila@nvidia.com >
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com >
Co-authored-by: Or Ozeri <or@ozery.com >
2026-09-24 07:09:12 +03:00
Jhao-Ting Chen
4fb767ff0a
[Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection ( #53175 )
...
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com >
2026-09-23 20:29:06 -07:00
e78e367c6e
[ROCm] Give turboquant boundary layers a layout-compatible backend ( #54988 )
...
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 22:00:07 -05:00
e6dc16cebd
[PCP] Support prefill context parallelism with data parallelism ( #57075 )
...
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Robert Shaw
6697a7dd30
[Scheduler] Tune --long-prefill-token-threshold adaptiveness ( #58459 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
2026-09-23 22:23:18 -04:00
e9f167129c
[CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs ( #58369 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 21:01:58 -05:00
Bob Ok and Wentao Ye
bcdacfc1ff
[Test][Determinism] Cover chunked prefill in the batch-invariance suite ( #55612 )
...
Signed-off-by: Bob Ok <49168652+blipbyte@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-23 21:21:56 -04:00
Juntian Liu and Claude Opus 5.5
9ef37771be
[Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap ( #58215 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 00:54:27 +00:00
Aarushi Jain and Andreas Karatzas
e0f03eb4f4
[CI] Share BF16 baselines across quantization comparison tests ( #58469 )
...
Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 19:26:09 -05:00
Kevin H. Luu and Claude Opus 5.5
7ef0d69ea0
[5/12][ci-selector][CI] Skip the Proton GPU test when another CUPTI tool is injected ( #58455 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 17:25:41 -07:00
Nick Hill
310f15d354
[Bugfix][MRV2] Align dummy idx_mapping dtype to avoid runtime jit ( #58462 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-23 16:54:48 -07:00
Misha Goin
89d9fd83e2
[CI] Disable JIT warmup by default in VllmRunner ( #58452 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-23 18:53:58 -05:00
Kevin H. Luu and Claude Opus 5.5
21ee743f47
[CI][Bugfix] Limit MRV2 sampler JIT warmup registration to ROCm ( #58465 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 16:11:50 -07:00
djramic and Andreas Karatzas
5fcc6e7c73
[ROCm][Bugfix] Fix TileLang mHC fused RMSNorm on 64-wide wavefronts ( #58419 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 23:03:45 +00:00
Andreas Karatzas
8b660ce96b
[ROCm][Test] Cover MoRI graph replay and output lifetime ( #58093 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 14:16:51 -07:00
陈志谦 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
97b1b12117
[Docs] Fix docstring typos (output_dytpe, kwrags, Abbrivations) ( #55936 )
...
Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 16:36:57 -04:00
Divy and Claude Opus 5
e26547adfb
[Bugfix] Pick a KV block size supported by every attention backend ( #49845 )
...
Signed-off-by: Divy <divy@coralbricks.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-23 16:36:35 -04:00
Thang Nguyen and Claude Opus 5.5
34b069080d
[CI] Select one GPU for the H200 initialized snapshot E2E step ( #58351 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-23 13:08:51 -07:00
Lio Einaudi
711fc55c10
[Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M ( #57586 )
...
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com >
2026-09-23 14:39:38 -04:00
CBP and Misha Goin
f9ad9dd6b4
Doc: add DiffusionGemma to supported models ( #46466 )
...
Signed-off-by: Bruce <Bruce798858117@gmail.com >
Signed-off-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-23 17:14:30 +00:00
Simon Danielsson
b5ea0c7107
[Bugfix][ROCm] Fix startup OOM in AITER MLA FP8 prefill workspace sizing ( #57923 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-23 11:44:12 -05:00
Dao007forever and Codex
c4f6ce4874
[Bugfix][NIXL] Restore successful push completion reporting ( #58188 )
...
Signed-off-by: Dao Le <Dao007forever@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-23 09:39:04 -07:00
c4cfd7007b
[CI][ROCm] Add an MI355 Kimi-K3 unit test group ( #58012 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 11:25:51 -05:00
Andreas Karatzas and OpenAI Codex
8033bd08c1
[ROCm][CI] Validate Mooncake and NIXL prefill/decode accuracy ( #58095 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 11:16:31 -05:00
b715699df4
[ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE ( #50212 )
...
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com >
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 11:11:57 -05:00
fxmarty-amd and mgoin
0549e8d0ab
[Quark] Remove quark-specific silent online quantization ( #51800 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-23 11:35:30 -04:00
f69cc7c178
[Quantization][Bugfix] Bump humming-kernels to 0.1.16 ( #58054 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-23 08:22:45 -07:00
HDCharles
9f07d023d0
[Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors ( #46528 )
2026-09-23 10:23:11 -04:00
Ankit-Jaiswal-AMD and Douglas Lehr
88afb9dcac
[CPU] Adds support for fp32 attention sinks ( #56252 )
...
Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-09-23 08:55:09 -05:00
Shuolei Wang
9ed81581a8
[Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… ( #58051 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Signed-off-by: Shuolei Wang <948904026@qq.com >
2026-09-23 21:03:21 +08:00
d95a896ea6
[Feature][Frontend] Add DeepSeek-V4 FIM completion rendering ( #44229 )
...
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 13:02:00 +00:00
157bcb7c48
[DSpark] Support pipeline-parallel targets in aggregated serving ( #56956 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-09-23 04:52:28 -07:00
15ed1262e7
[Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine ( #49648 )
...
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 10:37:40 +00:00
Tianyu Guo
c961121519
[Bugfix] Disable prefix caching for encoder-only before model config hooks ( #58287 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-09-23 09:25:32 +00:00
Andreas Karatzas
a22c637ed9
[ROCm][Test] Check GDN prefill numerics and output ownership ( #58091 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 02:19:58 -07:00
Yan Ma and Kunshang Ji
e34489a543
[XPU] upgrade to PyTorch 2.14 ( #56013 )
...
Signed-off-by: Yan Ma <yan.ma@intel.com >
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-23 09:18:52 +00:00
Gabriel Wu and Kimi Code
c843f0cab7
[Bugfix][SM120][MLA] Support NoPE sparse MLA (GLM-5.3-Flash) on the FlashInfer SM120 backend ( #55277 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-23 02:03:20 -07:00
Nick Hill
afec265bca
[MRV2] Miscellaneous code cleanup ( #57980 )
2026-09-23 01:51:52 -07:00
Andreas Karatzas and OpenAI Codex
36bc694889
[ROCm][CI] Include Python tooling in ROCm CI artifacts ( #58271 )
...
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-23 16:44:54 +08:00
Andreas Karatzas
56b3acb55b
[ROCm][Bugfix] Keep zero MiniMax MXFP8 activation blocks finite ( #58089 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 16:42:33 +08:00
1904ed9402
[Perf][ROCm][Attention] Narrow the Triton prefill-attention KV tile on RDNA3/RDNA4 ( #58225 )
...
Signed-off-by: Jipeng Li <jipengli@amd.com >
Co-authored-by: GitHub Copilot CLI <noreply@github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-23 08:39:27 +00:00
vllm-agent and kimi-no-na-wa
955bd6abef
[CI][Bugfix] Extend groupwise rms_norm scale tolerance to CUDA ( #58252 )
...
Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com >
2026-09-23 01:29:11 -07:00