 Nishant RevurandClaude
|
581692cc0f
|
[Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters (#57633)
Signed-off-by: Nishant Revur <kumrfni@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-09 01:44:50 +00:00 |
|
|
Misha GoinandClaude
|
e3a29aa9b3
|
[Bugfix][MoE] Size the permute scratch from its inputs (#60447)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-08 15:32:07 -04:00 |
|
|
Misha GoinandClaude
|
30c7405890
|
[MoE] MXFP4 oracle: add CUTLASS W4A4 backend and BF16 activation fallback (#60079)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-08 11:39:09 -07:00 |
|
|
|
d8fa463e2b
|
[Bugfix][Prefix caching] Fix inconsistent hybrid Mamba prefix cache boundaries (#60533)
Signed-off-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Wei Zhao <weizha@aws-cmh-slurm-1-login-01.cm.cluster>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-08 18:33:48 +00:00 |
|
|
|
9f3d70e081
|
[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 12:04:46 -04:00 |
|
|
JartX
|
994b8b77be
|
[Bugfix][ROCm] RDNA3 W4A16 MoE: handle the N-first CT weight layout (#60388)
Signed-off-by: JartX <sagformas@epdcenter.es>
|
2026-10-08 15:06:39 +00:00 |
|
|
Matthew BonanniandClaude Opus 5.5
|
4ed4c4433b
|
[Bugfix][HiSparse] Drop the unaccounted, write-only prefill mirror staging buffer (#60429)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 14:00:49 +00:00 |
|
|
Harry MellorandClaude Opus 5.5
|
21d9758459
|
Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 02:30:52 -07:00 |
|
|
liuzhenweiandKunshang Ji
|
0004536dd0
|
[XPU][Graph] Replace boolean-mask assignment with torch.where (#60529)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-10-08 17:13:48 +08:00 |
|
|
Paweł Mańczak
|
4cf1307e72
|
[XPU] Add tuned Mamba SSU configs for 2 more B70 shapes (#57565)
Signed-off-by: pmanczak <pawel.manczak@intel.com>
|
2026-10-08 16:36:05 +08:00 |
|
|
stefankoncarevicandAndreas Karatzas
|
814301b268
|
[Bugfix][Quant] Read safetensors headers of checkpoints with other file names (#60411)
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-10-08 01:38:41 -05:00 |
|
|
Jiahan Chang (Cyrus)
|
816173ca9d
|
[Kernel] Remove WAR for LL CuTeDSL BF16 GEMM (#60434)
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
|
2026-10-08 05:38:07 +00:00 |
|
|
|
81198e97ba
|
[Refactor] Abstract DeepGemm MegaMOE backend (#58053)
Signed-off-by: Hui Shi <kalinshi@tencent.com>
Co-authored-by: LuneExplorer <liyihan23@mails.tsinghua.edu.cn>
Co-authored-by: ArmageddonKnight <bojian@cs.toronto.edu>
|
2026-10-08 05:05:39 +00:00 |
|
|
|
775792f2d5
|
[Bugfix] Reland #53020: tie lm_head.weight for Nemotron Parse when checkpoint omits it (#60272)
Signed-off-by: Anish Kumar <aniskumar@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-10-08 04:40:55 +00:00 |
|
|
Isotr0pyandShanshan Shen
|
8352b2427f
|
[Multimodal] Optimize encoder cudagraph padding logic (#58925)
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Shanshan Shen <467638484@qq.com>
|
2026-10-08 03:38:09 +00:00 |
|
|
MattandClaude Opus 5.5
|
9dba339d07
|
[Bugfix][ROCm] Keep MLA derived weights in place on weight reload (#60389)
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
2026-10-07 20:38:06 -05:00 |
|
|
|
f38c6778ce
|
[quantization] Add per-token NVFP4 quantization MoE support for ReLU2 (#56740)
Signed-off-by: Xuanteng Huang <xuantengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-07 22:01:09 +00:00 |
|
|
|
b803edbc9f
|
[Model] Streamline EmbeddingGemma 2 config resolution and harmonize Triton prefill attention (#60289)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-10-07 14:56:40 -07:00 |
|
|
Misha GoinandClaude Opus 5.5
|
a982c81e3e
|
[Bugfix] Limit GlmMoeDsaForCausalLM fp8 KV cache default to SM100 (#60281)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 13:37:13 -07:00 |
|
|
|
e572e02d21
|
[UX] Skip checkpoint shards an MTP head does not load (#60152)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 15:16:19 -04:00 |
|
|
Jinzhen LinandKevin H. Luu
|
32fbfa15e8
|
[Bugfix][Quantization] Accept folded NVFP4 scales for Humming A16 (#60337)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
|
2026-10-07 11:04:43 -07:00 |
|
|
|
fba462e008
|
[Bugfix]Fix FlashInfer warmup crash: autotune(tuning_buckets=...) must round M up, not down (Invalid MXFP8 split-K tactic with spec-decode) (#58165)
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 14:01:21 -04:00 |
|
|
pmanczak
|
f7f62d1916
|
[XPU][MoE] Tune Triton fused MoE for Intel XPU (#53065)
|
2026-10-08 00:02:13 +08:00 |
|
|
xaguilar-amdandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
9367d8b969
|
[Bugfix][MLA] Scope DSpark non-causal capability to builder layers (#57992)
Signed-off-by: Xavier Aguilar <xavier.aguilarfruto@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-07 10:22:32 -05:00 |
|
|
Mikko TukiainenandCursor
|
548597367e
|
[ROCm][Perf] Add AITER FlyDSL GDN prefill backend (#57560)
Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-10-07 09:44:22 -05:00 |
|
|
|
c83935b300
|
[Bugfix][Model Loader] Support reload_weights with runai_streamer load format (#58124)
Signed-off-by: Viktor Koukouliev <vkoukouliev@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-07 14:04:35 +00:00 |
|
|
00CC
|
5281e49908
|
[ROCm][Perf] Enable medium-skinny dispatch in RDNAHybridW4A16LinearKernel (#52619)
Signed-off-by: tangzzycc <3081129260@qq.com>
|
2026-10-07 13:21:43 +00:00 |
|
|
SeongJun Lee
|
6b75dfb8a1
|
[Misc] Report a missing DeepSelect extension only when it is requested (#60242)
Signed-off-by: lesj0610 <lesj0610@gmail.com>
|
2026-10-07 09:53:17 +00:00 |
|
|
Robert ShawandClaude Code
|
283d76d56a
|
[Bugfix] Widen ep_gather output offset to int64 (#58373)
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
|
2026-10-07 01:45:04 +00:00 |
|
|
|
5ab4e4445b
|
[Perf][MoE] Skip padded work in block-FP8 DeepGEMM experts (#59128)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-10-06 18:49:57 -05:00 |
|
|
|
d1f3d8b870
|
[Quantization] Prefer Humming before Marlin backends on SM90 (#56997)
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 12:31:35 -07:00 |
|
|
Roger WangandKimi Code
|
bb87d227d4
|
[Bugfix][Model] Fix mypy safe-super error in EmbeddingGemma2Model (#60280)
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
|
2026-10-06 15:10:00 -04:00 |
|
|
|
3403e0f176
|
[Bugfix] Persist FlashInfer autotune cache per rank to fix the rank-0-only cache-hit deadlock (#57635)
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-10-06 17:51:01 +00:00 |
|
|
Luciano MartinsandLuciano Martins
|
02b83919aa
|
[Model] Support EmbeddingGemma2 multimodal pooling architecture (#60254)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
|
2026-10-06 13:43:38 -04:00 |
|
|
Wentao Ye
|
fba31f31a9
|
[GLM5.3 Perf] Switch to fp8 kv cache by default for GLM, 2.3%~5.5% E2E Throughput Improvement (#60140)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-10-06 12:54:20 -04:00 |
|
|
|
f8d92cae41
|
[Quantization] Layer re-quantization for linear layers through online quantization API (MXFP8 -> FP8 PTPC showcase) (#55684)
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: Tan Pin Siang <tanpinsiang@gmail.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-06 11:38:18 -05:00 |
|
|
Chris LeonardandShengqi Chen
|
cc73cca9c8
|
[Build] Migrate vendored DeepGEMM from pybind to TORCH_LIBRARY (abi3) (#48962)
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-10-06 08:39:04 -07:00 |
|
|
Samuel NordmannandMisha Goin
|
4a99ca40d6
|
[Perf][MoE] Eliminate staging from NCCL symmetric reduce-scatter (#49194)
Signed-off-by: samnordmann <snordmann@nvidia.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-06 07:37:01 -07:00 |
|
|
Matthias Gehre
|
b267fe4881
|
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides (#56301)
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
|
2026-10-06 08:22:33 -05:00 |
|
|
Mia Ojeda
|
3313825983
|
[Bugfix][DiffusionGemma] Read the causal mask as int32 and compile the sample step (#59992)
Signed-off-by: ojeda-e <eojeda@league.com>
|
2026-10-06 19:25:51 +08:00 |
|
|
aoshen02andClaude Opus 5.5
|
31e2443c90
|
[Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep (#60080)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-06 08:47:02 +00:00 |
|
|
roikoren755andzjy0516
|
e11962fc1c
|
[Feat][Mamba2] Enable internal prefill checkpoints (#57329)
Signed-off-by: Roi Koren <roik@nvidia.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-10-06 07:53:51 +00:00 |
|
|
|
edfccc9469
|
[ROCM][Perf] Add ROCM_SEGMENTED_ATTN attention-backend works across RDNA3/3.5/4 (#59132)
Signed-off-by: root <jeffaw99@hotmail.com>
Signed-off-by: big-yellow-duck <jeffaw99@hotmail.com>
Co-authored-by: Tan Pin Siang <1716735+tanpinsiang@users.noreply.github.com>
Co-authored-by: vllmellm <190700713+vllmellm@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-06 07:53:47 +00:00 |
|
|
|
40c86ae1ce
|
[Kernel] Enable OAI Triton MXFP4 MoE on SM12x (#58877)
Signed-off-by: lazymio <mio@lazym.io>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
|
2026-10-06 01:49:13 -04:00 |
|
|
|
21d93d0d8c
|
[Attention][Model] FP8 KV cache for Triton DiffKV and MiMo-V2.6-Flash (#58128)
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
|
2026-10-05 21:18:29 -07:00 |
|
|
Tianyu GuoandOpenAI Codex
|
8c417a2dae
|
[Bugfix][Model] Fix Qwen3-Omni interleaved M-RoPE boundary positions (#59842)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-06 03:35:53 +00:00 |
|
|
|
b2863c0de9
|
[Kimi-K3] Use the fused K/V pack kernel on the decode-context-parallel prefill path (#56881)
Signed-off-by: Brandon Zhang <31413216+brandonfzhang@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
|
2026-10-06 00:13:55 +00:00 |
|
|
MasonandYongye Zhu
|
6dd813e0b7
|
[Bugfix][MoE] Allow MiniMax2 routing in TRTLLM BF16 monolithic backend (#59571)
Signed-off-by: Mason Pitt <bradley.b.pitt@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-10-05 16:24:14 -07:00 |
|
|
akii96
|
182247257f
|
[ROCm][Perf] Use a stacked hipBLASLt GEMM for the bf16x3 router (#52668)
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
|
2026-10-05 17:28:07 +00:00 |
|
|
Sohaib Ahmed
|
64cb683842
|
[Model] Add Nemotron 3.5 ASR transcription support (#59827)
Signed-off-by: Sohaib-Ahmed21 <sohaibahmed1919@gmail.com>
|
2026-10-05 16:28:09 +00:00 |
|