MOVZX and Hinata
835c44af15
[Qwen4Exp] Reinterpret FP8 E4M3 PLE as int8 for SM86 pinned-host lookup (PR #58856 )
...
The pinned-host PLE lookup Triton kernel only gathers and stores bytes,
but passing an fp8e4nv tensor to it fails on architectures without
native FP8 pointer support (e.g. SM86) with an error saying type
fp8e4nv is not supported (supported fp8 dtypes are fp8e4b15, fp8e5).
Since FP8 E4M3 and int8 both occupy one byte per element, reinterpret
the PLE weight view and the output tensor as int8 when the weight dtype
is float8_e4m3fn; the non-FP8 path is unchanged.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-27 09:54:05 +07:00
MOVZX and Hinata
758cc4f9e1
[Qwen3.5] Pass quant_config and prefix to embed_tokens in Qwen3_5Model
...
Apply the qwen3_5_quantized_embedding patch: construct the embedding
through the model's quant config so quantized (e.g. fp8/int8)
embeddings are supported, and give it a stable prefix for weight
loading.
Co-Authored-By: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-09-27 09:53:21 +07:00
Yongye Zhu and Claude Opus 5.5
77871126f9
[Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC boundary ( #58586 )
...
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 03:08:36 +08:00
Cyrus Leung
379e9a1ea8
[Security] Harden message sanitization ( #58832 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-26 16:28:32 +00:00
Kevin H. Luu
7d8c5fe9a9
[Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata ( #58499 )
2026-09-26 11:37:36 -04:00
Andreas Karatzas
0239bd2549
[CI] Stabilize batch submission in full CUDA graph tests ( #58810 )
2026-09-26 11:35:56 -04:00
Wentao Ye
3576691c42
[GLM5.3 Bug] Fix sparse indexer attn topk backend selection ( #58594 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-26 11:35:28 -04:00
Misha Goin
79ac28c5bd
[CI] Reduce CUDA graph mode test overhead ( #58749 )
2026-09-26 11:22:33 -04:00
Ashraf Bhuiyan
5113cf9d16
[Mypy] Fix mypy typing for Qwen and Qianfan models ( #58046 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
2026-09-26 14:31:51 +00:00
Robert Shaw and Robert Shaw
dfab504333
[Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template ( #58754 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-26 14:00:56 +00:00
Robert Shaw and Robert Shaw
d64f277c54
[Bugfix] Fix Anthropic Thinking Disabled with P/D ( #58786 )
...
Signed-off-by: Robert Shaw <robshaw@redhat.com >
Co-authored-by: Robert Shaw <robshaw@redhat.com >
2026-09-26 13:50:57 +00:00
Willian Z and mgoin
ad6817b68d
[Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weights ( #58720 )
...
Signed-off-by: Willian <willian@willian.email >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-26 06:48:24 -07:00
Wentao Ye
bf93e1f92c
[Refactor] Remove dead tests utils ( #58803 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-26 09:16:53 -04:00
Itay Alroy and Tyler Michael Smith
2bb7605ae1
[Elastic EP] Fix EPLB load statistics during scaling ( #58473 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-09-26 12:05:37 +00:00
Juan Pérez de Algaba and Cursor
31f2e70cd3
[Security] Gate per-request multimodal processor kwargs ( #58830 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-26 11:10:35 +00:00
2f41e00235
[CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc named jobs ( #58609 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-09-26 04:10:14 -07:00
a4eb3f25d6
[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashInfer ( #57263 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-26 17:01:17 +08:00
Aarushi Jain and Andreas Karatzas
3b4566c5cf
[CI] Only isolate the registry tests that need a fresh process ( #58764 )
...
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-26 14:03:45 +08:00
ddd6fbca14
[Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings ( #58316 )
...
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: zhec <chengyunfei@ruc.edu.cn >
2026-09-26 02:18:58 +00:00
xiao-llm and Cursor
5840d95284
[Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support ( #57071 )
...
Signed-off-by: Xiao Yu <xiao.yu.dc@outlook.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-25 20:51:18 -05:00
b6d2b4fc25
[RL] Add sharding-aware NCCL M2N weight transfer ( #51520 )
...
Signed-off-by: Ke Wen <kwen@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-26 01:38:53 +00:00
Shuolei Wang
4bb804cc5b
[Perf][DSv4.1] Shard the Engram wkv projection across TP ranks ( #58678 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
2026-09-26 01:15:57 +00:00
Chandra Mourya and VS Chandra Mourya
8a2364605c
[Core] Bound draft-token RPC waits by the execute-model timeout ( #58779 )
...
Signed-off-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com >
Co-authored-by: VS Chandra Mourya <219748331+vschandramourya@users.noreply.github.com >
2026-09-26 00:40:22 +00:00
Baljinder Hothi and Flora Feng
fdcce47e9b
[Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning ( #58792 )
...
Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-26 00:38:52 +00:00
da1aaec31f
[CI] Split (H200 MIG/MI300) Basic Correctness into named jobs ( #57054 )
...
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-25 17:05:33 -07:00
Elvir Crnčević and Flora Feng
9bf44c4013
[Bugfix][Reasoning] Count Kimi K3 reasoning tokens ( #58372 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 23:29:56 +00:00
Rui Zhu and Claude
974cb65155
[Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_states ( #48419 )
...
Signed-off-by: Rui Zhu <rui.zhu.rz399@yale.edu >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-25 23:23:11 +00:00
Ricardo-M-L and Claude Opus 4.6
b040321b33
fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) ( #52580 )
...
Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-09-25 23:22:13 +00:00
Taneem Ibrahim and Wentao Ye
6ec2f84e1a
[Pooling] Preserve BERT-family heads for raw logits ( #57664 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-25 19:10:14 -04:00
Linkun
614b55f0b6
[AuxOutput] Only require Model Runner V2 on GPU platform ( #58205 )
...
Signed-off-by: Linkun Chen <github@lkchen.net >
2026-09-25 15:55:21 -07:00
Oxana Korzh
e55d076f89
[Bugfix] Don't drop the rest of the allocator config when toggling expandable segments ( #57982 )
...
Signed-off-by: Oxana Korzh <okorzh@amd.com >
2026-09-25 17:34:19 -05:00
4ccfe12398
[watermarking] golden tests for backwards compatibility ( #56809 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 22:01:48 +00:00
yzong-rh and Ben Browning
b6761e8ded
[Bugfix] Default missing detail for Responses API input images ( #57241 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
2026-09-25 14:08:55 -07:00
8b47e8b22c
[Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams ( #56067 )
...
Signed-off-by: Cheng Rui <286040359@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 14:08:23 -07:00
Nick Hill and Kimi Code
de2d478d9a
[Bugfix][Frontend] Document 404 response for /generative_scoring ( #58788 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-25 14:08:00 -07:00
7871963fcc
[Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop ( #58551 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 14:00:39 -07:00
Yousaf and sfeng33
83db8c8390
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 ( #55957 )
...
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-09-25 20:43:08 +00:00
e440b75bb6
[ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision ( #50605 )
...
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com >
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: jpvillam <juan.villamizar@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: jpvillam <juan.villamizar@amd.com >
2026-09-25 14:55:52 -05:00
31cc226401
[ROCm][CI] Run the MLA attention+quant fusion test on ROCm ( #58717 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 14:45:32 -05:00
stefankoncarevic and Andreas Karatzas
2d7dd0e5c1
[Bugfix][CI] Report subprocess test skips as skips, not passes ( #58701 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 14:44:49 -05:00
Taneem Ibrahim
2bc902eb0f
[CI] [MRV2] Restore MRV2 pp dp coverage ( #57735 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-25 19:13:59 +00:00
22bbe3f102
[ROCm][CI] Expand single-GPU coverage on MI355 DPX ( #57599 )
...
Signed-off-by: Sheral Kumar <shekumar@amd.com >
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com >
2026-09-25 13:27:53 -05:00
Matt Mastracci and Claude Fable 5.1
353e491efd
[Perf] DiffusionGemma: one-pass sampler statistics kernel ( #58226 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-25 14:14:55 -04:00
e3dd5b5a75
[ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch reference ( #58740 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 18:05:54 +00:00
1b922ffcbe
[ROCm][CI] Add quantized MoE serving test for gfx950 ( #58748 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 18:02:15 +00:00
b21757555a
[ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 ( #58724 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 17:43:40 +00:00
1417022c3d
[ROCm][Kimi-K3] Optimize low-concurrency speculative KDA ( #58045 )
...
Signed-off-by: jiacao-amd <jiahui.cao@amd.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-25 12:33:30 -05:00
GokayAI and GokayAI
c89b5b4176
[Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated head ( #58292 )
...
Signed-off-by: GokayAI <60583610+gokay-ai@users.noreply.github.com >
Co-authored-by: GokayAI <gokay-ai@users.noreply.github.com >
2026-09-25 17:23:44 +00:00
Wentao Ye
c88f4824ff
[GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level performance improvement ( #58450 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-25 13:15:18 -04:00
2617fe9383
[Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding ( #58454 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 12:51:34 -04:00