Juan Pérez de Algaba and Cursor
31f2e70cd3
[Security] Gate per-request multimodal processor kwargs ( #58830 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-26 11:10:35 +00:00
2f41e00235
[CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs ( #58609 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-26 04:10:14 -07:00
a4eb3f25d6
[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer ( #57263 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-26 17:01:17 +08:00
Aarushi Jain and Andreas Karatzas
3b4566c5cf
[CI] Only isolate the registry tests that need a fresh process ( #58764 )
...
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-26 14:03:45 +08:00
ddd6fbca14
[Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings ( #58316 )
...
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: zhec <chengyunfei@ruc.edu.cn >
2026-09-26 02:18:58 +00:00
xiao-llm and Cursor
5840d95284
[Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support ( #57071 )
...
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-25 20:51:18 -05:00
b6d2b4fc25
[RL] Add sharding-aware NCCL M2N weight transfer ( #51520 )
...
Signed-off-by: Ke Wen <kwen@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-26 01:38:53 +00:00
Shuolei Wang
4bb804cc5b
[Perf][DSv4.1] Shard the Engram wkv projection across TP ranks ( #58678 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-09-26 01:15:57 +00:00
Chandra Mourya and VS Chandra Mourya
8a2364605c
[Core] Bound draft-token RPC waits by the execute-model timeout ( #58779 )
...
Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com >
Co-authored-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com >
2026-09-26 00:40:22 +00:00
Baljinder Hothi and Flora Feng
fdcce47e9b
[Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning ( #58792 )
...
Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-26 00:38:52 +00:00
da1aaec31f
[CI] Split (H200 MIG/MI300) Basic Correctness into named jobs ( #57054 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-25 17:05:33 -07:00
Elvir Crnčević and Flora Feng
9bf44c4013
[Bugfix][Reasoning] Count Kimi K3 reasoning tokens ( #58372 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 23:29:56 +00:00
Rui Zhu and Claude
974cb65155
[Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states ( #48419 )
...
Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-25 23:23:11 +00:00
Ricardo-M-L and Claude Opus 4.6
b040321b33
fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) ( #52580 )
...
Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-09-25 23:22:13 +00:00
Taneem Ibrahim and Wentao Ye
6ec2f84e1a
[Pooling] Preserve BERT-family heads for raw logits ( #57664 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-25 19:10:14 -04:00
Linkun
614b55f0b6
[AuxOutput] Only require Model Runner V2 on GPU platform ( #58205 )
...
Signed-off-by: Linkun Chen <github@lkchen.net >
2026-09-25 15:55:21 -07:00
Oxana Korzh
e55d076f89
[Bugfix] Don't drop the rest of the allocator config when toggling expandable segments ( #57982 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
2026-09-25 17:34:19 -05:00
4ccfe12398
[watermarking] golden tests for backwards compatibility ( #56809 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 22:01:48 +00:00
yzong-rh and Ben Browning
b6761e8ded
[Bugfix] Default missing detail for Responses API input images ( #57241 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
2026-09-25 14:08:55 -07:00
8b47e8b22c
[Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams ( #56067 )
...
Signed-off-by: Cheng Rui <286040359@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 14:08:23 -07:00
Nick Hill and Kimi Code
de2d478d9a
[Bugfix][Frontend] Document 404 response for /generative_scoring ( #58788 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-25 14:08:00 -07:00
7871963fcc
[Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop ( #58551 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 14:00:39 -07:00
Yousaf and sfeng33
83db8c8390
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 ( #55957 )
...
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-09-25 20:43:08 +00:00
e440b75bb6
[ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision ( #50605 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: jpvillam <juan.villamizar@amd.com >
2026-09-25 14:55:52 -05:00
31cc226401
[ROCm][CI] Run the MLA attention+quant fusion test on ROCm ( #58717 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 14:45:32 -05:00
stefankoncarevic and Andreas Karatzas
2d7dd0e5c1
[Bugfix][CI] Report subprocess test skips as skips, not passes ( #58701 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 14:44:49 -05:00
Taneem Ibrahim
2bc902eb0f
[CI] [MRV2] Restore MRV2 pp dp coverage ( #57735 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-25 19:13:59 +00:00
22bbe3f102
[ROCm][CI] Expand single-GPU coverage on MI355 DPX ( #57599 )
...
Signed-off-by: Sheral Kumar <shekumar@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
2026-09-25 13:27:53 -05:00
Matt Mastracci and Claude Fable 5.1
353e491efd
[Perf] DiffusionGemma: one-pass sampler statistics kernel ( #58226 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-25 14:14:55 -04:00
e3dd5b5a75
[ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference ( #58740 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 18:05:54 +00:00
1b922ffcbe
[ROCm][CI] Add quantized MoE serving test for gfx950 ( #58748 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 18:02:15 +00:00
b21757555a
[ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 ( #58724 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 17:43:40 +00:00
1417022c3d
[ROCm][Kimi-K3] Optimize low-concurrency speculative KDA ( #58045 )
...
Signed-off-by: jiacao-amd <jiahui.cao@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 12:33:30 -05:00
GokayAI and GokayAI
c89b5b4176
[Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head ( #58292 )
...
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com >
Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com >
2026-09-25 17:23:44 +00:00
Wentao Ye
c88f4824ff
[GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement ( #58450 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-25 13:15:18 -04:00
2617fe9383
[Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding ( #58454 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 12:51:34 -04:00
Yashasvi Chaurasia
f83ea61cfe
[Benchmark] Record model_id in bench latency/throughput --output-json ( #58112 )
...
Signed-off-by: yashasvi <yashasvi@ibm.com >
2026-09-25 16:43:37 +00:00
2edbb32e58
[Bugfix][GLM-5.3-Flash] SM90 sparse MLA: index_kpool mismatch leads to corruption via unread query token ( #58704 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-09-25 12:36:36 -04:00
f339e0e757
[Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks ( #58583 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com >
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 16:33:19 +00:00
6491f481a7
[Bugfix] Stop allocator fragmentation from shrinking the KV cache during memory profiling ( #58430 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com >
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Robert Shaw <robertgshaw2-redhat@h100-01.nemg-001.lab.rdu2.dc.redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 12:16:27 -04:00
Mark McLoughlin
a44d7b5151
[Core][Logging] Fix JSON logging process decoration ( #57957 )
...
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
2026-09-25 10:12:28 -06:00
38cc054fc2
[Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 quantization ( #58194 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-25 09:19:48 -06:00
chuan932 and Misha Goin
f550ad8bf7
[Kernel][Perf] Register-resident path for per-token-group 8-bit quant ( #55330 )
...
Signed-off-by: chao.huan <chao.huan@nio.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-09-25 08:11:42 -07:00
Taneem Ibrahim and Wentao Ye
bb35e2587b
[Pooling] Preserve reranker tokenization with document limits ( #57666 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-25 11:04:41 -04:00
Taneem Ibrahim and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
50f78084da
[Bugfix] Fix generative scoring body cancellation ( #57729 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 11:01:31 -04:00
Nick Hill
234df712ed
[MRV2] Minor model_runner.py code cleanup ( #58610 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-25 07:59:27 -07:00
73a78e6f1f
[SpecDecode] Add LiLiCorr drafter ( #57934 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Signed-off-by: Andrii Skliar <andreyws96@gmail.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-09-25 10:08:28 -04:00
djramic and Andreas Karatzas
25b0add7b8
[ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests ( #58698 )
...
Signed-off-by: Djordje Ramic <djoramic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 10:52:15 +00:00
Mark McLoughlin and AI Assistant
45d56c0baf
[Core] Model console logging as CLI configuration ( #57205 )
...
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: AI Assistant <noreply@openai.com >
2026-09-25 11:36:59 +01:00
Thang Nguyen and Claude Opus 5.5
fa6c407d15
[CI] Shard (H100) Helion Kernels five ways ( #58645 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-25 03:25:11 -07:00
