Mohammad Hijjawi and Cyrus Leung
ef014bdf06
[Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses ( #59072 )
...
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-07 04:13:52 +00:00
Hyukjoon Lee and Claude Opus 5.5
2342343746
[ROCm][DCP] Default MLA DCP verify to the round-robin asm decode where it applies ( #59965 )
...
Signed-off-by: hyukjlee <hyukjlee@amd.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 03:10:54 +00:00
8a26869bca
[Bugfix] Index Mamba align state copies by request slot in MRV2 ( #60210 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Karl0007 <44745209+Karl0007@users.noreply.github.com >
Co-authored-by: Yongqin Wang <yongqinwang@roblox.com >
2026-10-07 02:32:46 +00:00
956960f85f
[Perf][MRV2] Avoid stream syncs in NonUvaBuffer host-to-device copies ( #60215 )
...
Signed-off-by: cheesecake <farzanaman99@gmail.com >
Co-authored-by: William Joyner <133391209+joynerwk03@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 01:51:12 +00:00
Robert Shaw and Claude Code
283d76d56a
[Bugfix] Widen ep_gather output offset to int64 ( #58373 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-07 01:45:04 +00:00
fe51194e2d
[Bugfix][MiniMax-M3] Fence decode indexer shared-memory stage reuse ( #60194 )
...
Signed-off-by: Vladimir Anisimov <vanisimov@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
2026-10-07 01:35:12 +00:00
Miłosz Grunwald
703c29179c
[TEST][XPU][CI] Generalize model runner prefill tail classification tests to all platforms ( #60201 )
...
Signed-off-by: microslaw <milosz.grunwald@intel.com >
2026-10-07 08:59:20 +08:00
Sriraj and Flora Feng
db6e3cd8c4
[Bugfix][Frontend] Log the full request body when max_log_len is unset ( #60310 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:42:17 +00:00
jxj and OpenAI Codex
6c2925fe5f
[Bugfix][Core] Preserve speculative metrics when coalescing outputs ( #55708 )
...
Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-07 08:35:14 +08:00
Sriraj and Flora Feng
d77cf81156
[Bugfix][Frontend] Honor default penalties in batch chat completions ( #60307 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:21:00 +00:00
5ab4e4445b
[Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts ( #59128 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 18:49:57 -05:00
ae19af883a
[CI] Require transformers 5.19 for the EmbeddingGemma2 registry test ( #60295 )
...
Signed-off-by: khluu <khluu000@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-10-06 14:50:10 -07:00
aditi-amd
385d86a6cf
[Attention] Add UltraQuant 4-bit KV cache backend (FlyDSL D=256) ( #57057 )
...
Signed-off-by: Aditi Rana <aditi.rana@amd.com >
2026-10-06 20:42:52 +00:00
819852df6b
[ModelRunner V2] Speculative Decoding NGram GPU Implementations ( #40704 )
...
Signed-off-by: PatchouliTaisa <patchychen@tencent.com >
Signed-off-by: PatchouliTaisa <pyramkar@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: PatchouliTaisa <patchychen@tencent.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Xuanan Chen <xuananchenc@nvidia.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
2026-10-06 13:32:59 -07:00
Benjamin Chislett and Claude Opus 5.5
dac35c5b66
[Misc] Add chat-completion options to gsm8k_eval.py CLI ( #60114 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 20:13:44 +00:00
d1f3d8b870
[Quantization] Prefer Humming before Marlin backends on SM90 ( #56997 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 12:31:35 -07:00
e37e51dd24
[Metrics] Expose cached prompt tokens by cache tier ( #56318 )
...
Signed-off-by: Cam Quilici <cjquilici@gmail.com >
Signed-off-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Cam Quilici <cameron@semianalysis.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
2026-10-06 14:52:05 -04:00
Nishant Revur and Claude
ad0f67a7ec
[Bugfix][Spec Decode] Keep heterogeneous-vocab draft models on Model Runner V1 ( #59541 )
...
Signed-off-by: Nishant Revur <kumrfni@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 18:03:44 +00:00
3403e0f176
[Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock ( #57635 )
...
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com >
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com >
2026-10-06 17:51:01 +00:00
Luciano Martins and Luciano Martins
02b83919aa
[Model] Support EmbeddingGemma2 multimodal pooling architecture ( #60254 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
2026-10-06 13:43:38 -04:00
7b665f7a58
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) ( #58181 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 17:22:47 +00:00
f8d92cae41
[Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) ( #55684 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com >
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 11:38:18 -05:00
eky-amd and Claude Fable 5
29f955cce0
[ROCm] Fuse MLA dual RMSNorm + FP8 group quant for DeepSeek-R1 ( #54857 )
...
Signed-off-by: Ethan Ky <ethan.ky@amd.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-06 11:26:38 -05:00
stefankoncarevic
4516896626
[CI/Build] Warm the DP engines before measuring load balance in test_load ( #60241 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-10-06 11:23:37 -05:00
Giancarlo Delfin
fcf53d240c
[Spec Decode] Remove eager metadata rebuild during MTP fused multi-step decode ( #58463 )
...
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
2026-10-06 12:17:30 -04:00
Samuel Nordmann and Misha Goin
4a99ca40d6
[Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter ( #49194 )
...
Signed-off-by: samnordmann <snordmann@nvidia.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-06 07:37:01 -07:00
Matthias Gehre
b267fe4881
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides ( #56301 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-06 08:22:33 -05:00
Artem Perevedentsev and Shashank Varma
049507aa76
[Bugfix][Structured Output] Flag multi-branch allOf as unsupported for xgrammar ( #59061 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Shashank Varma <324153016+shashankvarma499@users.noreply.github.com >
2026-10-06 17:01:04 +04:00
Sherif Waly
25385102fe
[Bugfix][KV Offload] Honor speculative cacheability in SimpleCPU ( #60071 )
...
Signed-off-by: Sherif Waly <sherif.waly@mistral.ai >
2026-10-06 14:25:08 +02:00
Mia Ojeda
3313825983
[Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step ( #59992 )
...
Signed-off-by: ojeda-e <eojeda@league.com >
2026-10-06 19:25:51 +08:00
09e0ce3d82
[Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server ( #54835 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-06 18:46:24 +08:00
roikoren755 and zjy0516
e11962fc1c
[Feat][Mamba2] Enable internal prefill checkpoints ( #57329 )
...
Signed-off-by: Roi Koren <roik@nvidia.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-10-06 07:53:51 +00:00
edfccc9469
[ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 ( #59132 )
...
Signed-off-by: root <jeffaw99@hotmail.com >
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com >
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com >
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-06 07:53:47 +00:00
15c1cc12ac
[Bugfix][EPLB] Use group-local ranks for torch P2P transfers ( #55804 )
...
Signed-off-by: LostFox11 <wangziyue17@huawei.com >
Co-authored-by: LostFox11 <wangziyue17@huawei.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: YangShuai52 <178856220+YangShuai52@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-06 07:44:57 +00:00
sungbin1015 and Claude
1e5d0ea888
[CI/Build] Fix incorrect CLI arg validation tests in test_cli_args.py ( #46779 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 05:58:47 +00:00
40c86ae1ce
[Kernel] Enable OAI Triton MXFP4 MoE on SM12x ( #58877 )
...
Signed-off-by: lazymio <mio@lazym.io >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Shengqi Chen <harry-chen@outlook.com >
2026-10-06 01:49:13 -04:00
Bugen Zhao and Claude Opus 5.5
b3082d3044
[Rust Frontend] Remove the PyO3 tool-parser bridge ( #59744 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:46:27 +00:00
Bugen Zhao and Claude Opus 5.5
5145e35e50
[Frontend] Port the MiniMax M3 tool parser to the parser engine ( #59743 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:46:27 +00:00
czhu-cohere and aoshen02
6fddc9b21e
[Profiler] Add per-session profiling controls to Python and Rust frontends ( #57875 )
...
Signed-off-by: czhu <conway.zhu@cohere.com >
Signed-off-by: Conway Zhu <conway.zhu@cohere.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-06 05:21:19 +00:00
aoshen02 and Claude Opus 5.5
9a1f6fd4cb
[Model][DeepSeek-V4.1] Permute mega-attention wq_b / wo_a while loading ( #60064 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:20:09 +00:00
Chinmay-Kulkarni-AMD and Cursor Agent
533d81b4b0
[ZenCPU][Test] Add zentorch/torch version pin consistency test ( #52888 )
...
Signed-off-by: Chinmay Kulkarni <Chinmay.Kulkarni@amd.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-10-06 13:12:19 +08:00
eb2901b735
[Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs ( #59532 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 22:11:45 -07:00
21d93d0d8c
[Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash ( #58128 )
...
Signed-off-by: Luca Motz <luca.motz@icloud.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
2026-10-05 21:18:29 -07:00
Tianyu Guo
30ab428235
[Bugfix][Attention] Restrict FlashInfer sparse MLA FULL graphs to decode ( #59751 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-06 04:13:23 +00:00
lzhan011 and Flora Feng
a3e0243b1c
[Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON ( #54844 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 03:37:30 +00:00
Tianyu Guo and OpenAI Codex
8c417a2dae
[Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions ( #59842 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-06 03:35:53 +00:00
Tianyu Guo
48eb44b719
[Bugfix][KVConnector][NIXL] Fix TP handshake test after per-region FA desc flags ( #60157 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-10-06 03:19:41 +00:00
6fa67afed9
[Bugfix][Core] Retire stale Mamba spec blocks in prefill checkpoint steps ( #59759 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-06 11:01:19 +08:00
2b32d9ac6a
[Bugfix][Core] Drop stale block hashes when a session truncates tokens ( #59103 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Co-authored-by: Brian Woods <me@brian.biz >
Co-authored-by: Change72 <changg@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-06 02:51:03 +00:00
9465e1c783
[Bugfix][NIXL] Recover locally invalidated pull peer metadata ( #55471 )
...
Signed-off-by: xijiade.aihemaiti <3146335281@qq.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Yannik Hinteregger <37209495+YannikHinteregger@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 18:46:53 -07:00
