Commit Graph
21968 Commits
Author SHA1 Message Date
Talor Abramovich 28282ffe34 [Feature][Frontend] Request JSON body debug logging on --enable-log-requests flag (#58163)
Signed-off-by: talora <talora@nvidia.com>
2026-09-24 08:08:40 +00:00
Thien TranandOpenAI Codex 00b7847c80 [Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512)
Signed-off-by: OpenAI Codex <codex@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-24 00:41:36 -07:00
1291bdb85e [EPD][Model Loader] Skip language-model checkpoint shards for --mm-encoder-only (#58086)
Signed-off-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: grYe99 <guorongye99@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-09-24 00:41:15 -07:00
Andreas KaratzasandOpenAI Codex c3f5270270 [ROCm][CI] Add the MI355 TurboQuant t3nc mirror (#58432)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
2026-09-24 02:16:09 -05:00
Tianmu LiOpenAI CodexLi, Jiang <jiang1.li@intel.com>
67ddac3ea8 [CPU][GDN] Support NIXL DS convolution-state layout (#53300)
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-24 14:55:20 +08:00
Andreas KaratzasandCodex 153f2ba1b9 [Compile][CI] Honor Triton cache overrides and add AMD timeout headroom (#58474)
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-24 01:27:07 -05:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e028554a1f [Rust Frontend] Support --sse-keep-alive-interval (#58306)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-24 06:19:14 +00:00
Biswa PandaandCodex cf78797cb7 [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634)
Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias.

Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-24 14:08:26 +08:00
lucamotzandClaude Opus 5.5 ee2f3cdf12 [Bugfix][Spec Decode] Separate DSpark width from MTP stage validation (#54631)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 06:08:22 +00:00
44bf011f44 [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-09-23 22:45:23 -07:00
Bugen ZhaoandClaude Opus 5.5 a1b6763df6 [Rust Frontend] Accept custom chat roles for HF templates (#58311)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-24 13:43:34 +08:00
Bugen ZhaoandClaude Opus 5.5 a5603f21ae [Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op (#58330)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-24 13:43:24 +08:00
Farzad AbdolhosseiniClaudeLi, Jiang <jiang1.li@intel.com>
8b06a343ec [Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 (#56168)
Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
2026-09-24 13:15:17 +08:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
2026-09-24 04:46:16 +00:00
Kapil AryaandOr Ozeri 70fc359d25 [Bugfix][KV Offload] Retain offload event metadata through batch translation (#57453)
Signed-off-by: Kapil Arya <kapila@nvidia.com>
Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com>
Co-authored-by: Or Ozeri <or@ozery.com>
2026-09-24 07:09:12 +03:00
Jhao-Ting Chen 4fb767ff0a [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175)
Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
2026-09-23 20:29:06 -07:00
e78e367c6e [ROCm] Give turboquant boundary layers a layout-compatible backend (#54988)
Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-23 22:00:07 -05:00
Lucas WilkinsonQiuChunshuoClaude Fable 5.1CodexRobert ShawNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e6dc16cebd [PCP] Support prefill context parallelism with data parallelism (#57075)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 22:32:36 -04:00
Robert Shaw 6697a7dd30 [Scheduler] Tune --long-prefill-token-threshold adaptiveness (#58459)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
2026-09-23 22:23:18 -04:00
e9f167129c [CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs (#58369)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-23 21:01:58 -05:00
Bob OkandWentao Ye bcdacfc1ff [Test][Determinism] Cover chunked prefill in the batch-invariance suite (#55612)
Signed-off-by: Bob Ok <49168652+blipbyte@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-09-23 21:21:56 -04:00
Juntian LiuandClaude Opus 5.5 9ef37771be [Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215)
Signed-off-by: Juntian Liu <Juntianl777@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 00:54:27 +00:00
Aarushi JainandAndreas Karatzas e0f03eb4f4 [CI] Share BF16 baselines across quantization comparison tests (#58469)
Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-23 19:26:09 -05:00
Kevin H. LuuandClaude Opus 5.5 7ef0d69ea0 [5/12][ci-selector][CI] Skip the Proton GPU test when another CUPTI tool is injected (#58455)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 17:25:41 -07:00
Nick Hill 310f15d354 [Bugfix][MRV2] Align dummy idx_mapping dtype to avoid runtime jit (#58462)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-09-23 16:54:48 -07:00
Misha Goin 89d9fd83e2 [CI] Disable JIT warmup by default in VllmRunner (#58452)
Signed-off-by: mgoin <mgoin64@gmail.com>
2026-09-23 18:53:58 -05:00
Kevin H. LuuandClaude Opus 5.5 21ee743f47 [CI][Bugfix] Limit MRV2 sampler JIT warmup registration to ROCm (#58465)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 16:11:50 -07:00
djramicandAndreas Karatzas 5fcc6e7c73 [ROCm][Bugfix] Fix TileLang mHC fused RMSNorm on 64-wide wavefronts (#58419)
Signed-off-by: Djordje Ramic <djoramic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-23 23:03:45 +00:00
Andreas Karatzas 8b660ce96b [ROCm][Test] Cover MoRI graph replay and output lifetime (#58093)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-09-23 14:16:51 -07:00
陈志谦andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 97b1b12117 [Docs] Fix docstring typos (output_dytpe, kwrags, Abbrivations) (#55936)
Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-23 16:36:57 -04:00
DivyandClaude Opus 5 e26547adfb [Bugfix] Pick a KV block size supported by every attention backend (#49845)
Signed-off-by: Divy <divy@coralbricks.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 16:36:35 -04:00
Thang NguyenandClaude Opus 5.5 34b069080d [CI] Select one GPU for the H200 initialized snapshot E2E step (#58351)
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 13:08:51 -07:00
Lio Einaudi 711fc55c10 [Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M (#57586)
Signed-off-by: LioEinaudi <zhao3024667639@gmail.com>
2026-09-23 14:39:38 -04:00
CBPandMisha Goin f9ad9dd6b4 Doc: add DiffusionGemma to supported models (#46466)
Signed-off-by: Bruce <Bruce798858117@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-09-23 17:14:30 +00:00
Simon Danielsson b5ea0c7107 [Bugfix][ROCm] Fix startup OOM in AITER MLA FP8 prefill workspace sizing (#57923)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
2026-09-23 11:44:12 -05:00
Dao007foreverandCodex c4f6ce4874 [Bugfix][NIXL] Restore successful push completion reporting (#58188)
Signed-off-by: Dao Le <Dao007forever@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-23 09:39:04 -07:00
c4cfd7007b [CI][ROCm] Add an MI355 Kimi-K3 unit test group (#58012)
Signed-off-by: Oxana Korzh <okorzh@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
2026-09-23 11:25:51 -05:00
Andreas KaratzasandOpenAI Codex 8033bd08c1 [ROCm][CI] Validate Mooncake and NIXL prefill/decode accuracy (#58095)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-09-23 11:16:31 -05:00
b715699df4 [ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE (#50212)
Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com>
Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
2026-09-23 11:11:57 -05:00
fxmarty-amdandmgoin 0549e8d0ab [Quark] Remove quark-specific silent online quantization (#51800)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
2026-09-23 11:35:30 -04:00
f69cc7c178 [Quantization][Bugfix] Bump humming-kernels to 0.1.16 (#58054)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
2026-09-23 08:22:45 -07:00
HDCharles 9f07d023d0 [Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors (#46528) 2026-09-23 10:23:11 -04:00
Ankit-Jaiswal-AMDandDouglas Lehr 88afb9dcac [CPU] Adds support for fp32 attention sinks (#56252)
Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
2026-09-23 08:55:09 -05:00
Shuolei Wang 9ed81581a8 [Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… (#58051)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
2026-09-23 21:03:21 +08:00
d95a896ea6 [Feature][Frontend] Add DeepSeek-V4 FIM completion rendering (#44229)
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-09-23 13:02:00 +00:00
157bcb7c48 [DSpark] Support pipeline-parallel targets in aggregated serving (#56956)
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-09-23 04:52:28 -07:00
15ed1262e7 [Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine (#49648)
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-09-23 10:37:40 +00:00
Tianyu Guo c961121519 [Bugfix] Disable prefix caching for encoder-only before model config hooks (#58287)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
2026-09-23 09:25:32 +00:00
Andreas Karatzas a22c637ed9 [ROCm][Test] Check GDN prefill numerics and output ownership (#58091)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-09-23 02:19:58 -07:00
Yan MaandKunshang Ji e34489a543 [XPU] upgrade to PyTorch 2.14 (#56013)
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-09-23 09:18:52 +00:00