Matt and Claude Opus 5
9163190dda
[ROCm][Bugfix][Perf] Tune multi-stream shared experts use; wvSplitKrc fixes ( #56098 )
...
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-10 15:14:22 -07:00
Micah Williamson
7de70fa7ae
[Bugfix][ROCm] Create linear layer biases with requires_grad=False ( #56161 )
...
Signed-off-by: Micah Williamson <micah.williamson@amd.com >
2026-09-10 13:54:52 -05:00
Simon Danielsson
7470082f57
[ROCm][Perf] Add bpreshuffled blockscaled fp8 GEMM ( #51692 )
...
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com >
2026-09-10 16:18:37 +00:00
Itay Alroy and Tyler Michael Smith
bdea2777ea
[Bugfix][MoE] Fix batched CUTLASS workspace overallocation ( #55579 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-09-10 15:33:56 +00:00
Alexander Crettenand and Claude
7d8d71e989
[Bugfix] Qwen3-VL(-MoE): pass architectures to with_hf_config for pipeline parallelism ( #43272 )
...
Signed-off-by: Xonder <xonder@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-10 14:49:35 +01:00
Taneem Ibrahim
f9083cb83a
[Mypy] Fix typing for R/S models ( #54157 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-10 09:43:08 -04:00
siyu and Isotr0py
73fb19151f
[Fast Start] Support fp4 ( #55465 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 13:22:17 +00:00
YoavMiron and Benjamin Chislett
40e6042ec8
[Feature][Spec Decode] MTP with separate (possibly quantized) lm head for nemotron ( #54574 )
...
Signed-off-by: Yoav Miron <yomiron@nvidia.com >
Signed-off-by: YoavMiron <yomiron@nvidia.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-09-10 08:26:31 -04:00
9b959b8657
[Model] DeepSeek-V4.1-Flash Model Definitions ( #56228 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-10 20:16:26 +08:00
Jakub Byczkowski and Copilot
c9355e25e8
[XPU][Bugfix] Add forward_xpu to Ernie4_5_VLRotaryEmbedding ( #55942 )
...
Signed-off-by: Jakub Byczkowski <jakub.byczkowski@intel.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-09-10 17:48:00 +08:00
6fd08c45e6
[XPU][Bugfix] Fix FalconH1 pipeline-parallel execution ( #55913 )
...
Signed-off-by: Jakub Byczkowski <jakub.byczkowski@intel.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-09-10 17:47:29 +08:00
fe26c705f7
[Model] Add Bailing V3 VL support ( #55921 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-10 02:46:56 -07:00
Jared Wen and Claude Fable 5.1
1768273c13
[Perf][GLM-5.3-Flash] Decode hot-path cleanups: strided KDA recurrent inputs, NoPE MQA query without concat, no duplicate router GEMM ( #55736 )
...
Signed-off-by: Jared Wen <jaredwen@inferact.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-10 16:28:33 +08:00
Yunzhu Lu and Isotr0py
442d36031c
[MM][CG] Enable encoder CUDA Graph for MiniCPM-V ( #42785 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 06:28:48 +00:00
Li, Jiang
c3ccc0e957
[Model] Add DeepSeek-V4 CPU backend ( #55355 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-09-10 14:14:48 +08:00
Qiming Zhang
49632fb3ac
[CI][XPU] Reject CUDA-IPC weight cache on non-CUDA/ROCm platforms ( #56010 )
...
Signed-off-by: mayuyuace <qiming1.zhang@intel.com >
2026-09-10 14:05:07 +08:00
Colin Z
588a813a60
[ROCm] [BugFix] Fix AITER MXFP4 ASM-GEMM crash on unfused shared experts ( #55213 )
...
Signed-off-by: Colin Zeng <Colin.Zeng@amd.com >
2026-09-10 05:23:36 +00:00
Wei Zhao and OpenAI Codex
8c87c333b8
[Perf] Fix TRTLLM ragged prefill perf regression ( #55499 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-09 20:38:20 +00:00
22d95d1adc
[Bugfix] Make mm_device_do_normalize encoder-cudagraph safe ( #55370 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 01:43:22 +08:00
83fe99399e
[Spec Decode] Add NVFP4 DSpark gathered top-k projection ( #55713 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 17:33:46 +00:00
Harry Mellor and Claude Opus 5
42d76ee35a
[Docs] Fix griffe docstring indentation warning in SupportsMRoPE ( #56112 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-09 10:11:40 -07:00
Harry Mellor and Claude Opus 5
719284fe15
[Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count ( #56078 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-09 09:23:30 -07:00
d8d53f17c2
[Rocm][Kimi-k3] Add pipeline_parallel support for the kimik3 model ( #53664 )
...
Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-09 07:50:24 -07:00
cc4210f671
[Perf] Improve BF16x3 router GEMM accuracy and make it default on sm100 ( #55899 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-09-09 07:33:12 -07:00
Harry Huang and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
3116c5d06b
[Qwen4Exp] Support UVA PLE-offload and Engram tensor parallelism ( #54371 )
...
Signed-off-by: huanghaoyan.hhy <huanghaoyan.hhy@alibaba-inc.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 07:32:34 -07:00
Thomas Ortner and Harry Mellor
e509d32b5a
[Transformers backend] Enable QKV-Fuser for Gemma4 ( #55690 )
...
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com >
Signed-off-by: Thomas Ortner <bohnstinglthomas@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-09 14:46:31 +01:00
abrahamzewoudie
08b3e67b66
[ROCm][Perf] Fix Qwen3-vLLM audio encoder TP when heads are not divisible by TP size ( #45900 )
...
Signed-off-by: abrahamzewoudie <azewoudi@amd.com >
2026-09-09 13:28:24 +00:00
Michael Lazos and OpenAI Codex
28a73ccfba
[Kernel] Add NVFP4 support to the torch linear backend ( #53319 )
...
Signed-off-by: Michael Lazos <mlazos@meta.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 20:30:05 -04:00
Nick Hill
4aadb0f14d
[Core] Enhance cpu<->gpu sync checking to include paged async copies ( #53491 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-09-08 17:26:21 -07:00
Taneem Ibrahim
5af4cc33ec
[Bugfix] Fix OpenPangu multimodal embedding merge ( #55941 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-08 23:53:59 +00:00
c0d8d5413e
[Transformers] Generalize merged-column linear fusion ( #55301 )
...
Signed-off-by: BANANASJIM <bananasjim1@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-08 23:49:37 +00:00
60ad959b6f
Fix block FP8 MTP in ModelOpt mixed checkpoints ( #55513 )
...
Signed-off-by: Shiyang Chen <shiychen@nvidia.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-08 22:47:21 +00:00
d29c88f162
[Core] Let SWA layers take the primary block size to avoid inflating the KV block LCM ( #53007 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-08 16:44:43 -04:00
Roderick Wu and mgoin
2c9d68f80b
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ( #54809 )
...
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com >
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 12:45:12 -07:00
8ebc5b0a18
[Bugfix] Fix NVFP4 fused SiLU+mul scale allocation and global scale direction ( #55643 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: jiahao <jxia77@terpmail.umd.edu >
Co-authored-by: Kimi Code <noreply@moonshot.cn >
2026-09-08 08:26:07 -07:00
Wentao Ye
6b15bea080
[Refactor] Remove utils dead code ( #53941 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-09-08 11:22:09 -04:00
Canlin Guo and OpenAI Codex
8e1f97e709
[Qwen3.8-Flash-Next] Tune FP8 TP2/TP4 Triton MoE on B200 ( #55890 )
...
Signed-off-by: Canlin <canlinguosdu@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-09-08 22:45:18 +08:00
Roberto L. Castro and Codex
6ddbab03de
[4/N][warmup][DSv4] Migrate common attention kernels ( #50176 )
...
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com >
Co-authored-by: Codex <codex@openai.com >
2026-09-08 10:25:48 -04:00
Mohit Singla and Codex
613ab20f7f
[Model] Enable torch.compile for Sarvam MLA ( #55817 )
...
Signed-off-by: mohit-sarvam <mohit@sarvam.ai >
Co-authored-by: Codex <noreply@openai.com >
2026-09-08 22:01:30 +08:00
414057a3d3
[Bugfix] Apply attention sinks in the Transformers backend ( #52156 )
...
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-09-08 13:54:07 +00:00
472d8c9125
[Perf] Fuse DeepEncoder relative bias in Triton attention ( #55629 )
...
Signed-off-by: levius <2114377220@qq.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-08 13:22:04 +00:00
Stefano Castagnetta
f6326f53bd
[Perf][GDN] Enable the FlashInfer GDN prefill kernel on SM12x ( #55715 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
2026-09-08 12:26:03 +00:00
Stefano Castagnetta
13cf9e05c1
[Perf][Quant][NVFP4] Prefer W4A4 linear kernels over weight-only ones on SM120/121 ( #55170 )
...
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com >
2026-09-08 15:06:45 +04:00
jimmy-adams
e41a17e606
[ROCm][Quantization] Support AMD Quark per-block FP8 for fused MoE layers ( #52263 )
...
Signed-off-by: huji <huidong.ji@amd.com >
Signed-off-by: jimmy-adams <41593649+jimmy-adams@users.noreply.github.com >
2026-09-08 10:30:26 +00:00
Taoyu Zhu and Shanshan Shen
ce6c241ddc
[ROCm][Perf][M3] Fused allreduce+GemmaRMSNorm fast path ( #54787 )
...
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com >
Co-authored-by: Shanshan Shen <467638484@qq.com >
2026-09-08 07:21:49 +00:00
Subhash Polisetti
782f36cd0c
[Bugfix][InternVL] Stop the video parser consuming image_embeds ( #55779 )
...
Signed-off-by: subhash polisetti <subhashr161347@gmail.com >
2026-09-08 05:19:41 +00:00
weihan
9ea8f3ffc3
[Bugfix] Honor STEP token pooling in DispatchPooler.for_seq_cls ( #55307 )
...
Signed-off-by: weihan <130119942+krsish@users.noreply.github.com >
2026-09-08 04:13:09 +00:00
7fa2c63796
[Bugfix] Autotune FlashInfer deferred MoE decode kernels before CUDA graph capture ( #55377 )
...
Signed-off-by: ming <ming@inferact.ai >
Co-authored-by: Kimi Code <kimi-code@moonshot.cn >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-08 04:02:23 +00:00
Wei Zhao
a69402aaa8
[Perf] Integrate FlashInfer KDA kernels ( #55364 )
...
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com >
2026-09-07 20:29:30 -07:00
leo
12579c7f01
[Kernel][Perf] Tune H20 block-FP8 MoE low-batch configs (+21%) ( #54668 )
...
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com >
2026-09-08 03:20:21 +00:00
