|
107a03ba63
|
[Core] Support fp32 lm_head for generation models via head_dtype (RFC #48305 §3.6) (#48390)
Signed-off-by: Karthik Kothuri <karthikkothuri2009@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-07-13 16:43:34 +08:00 |
|
|
|
bea70c7cfc
|
[Attention] Make sliding-window support an explicit backend capability (#48011)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-13 01:07:56 -07:00 |
|
|
Canlin Guo
|
36484e464a
|
[BugFix] Restore full tokens for Qwen MTP When MoE SP (#48429)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
|
2026-07-13 13:29:41 +08:00 |
|
|
Yejing Lai
|
8c5dafcd09
|
[Bugfix][UT]Fix EagleMiniCPMForCausalLM meet TypeError (#48452)
Signed-off-by: Lai, Yejing <yejing.lai@intel.com>
|
2026-07-13 04:37:23 +00:00 |
|
|
|
05fa8183a6
|
[CPU][Spec Decode] Support DFlash speculative decoding for GDN models on CPU (#46090)
Signed-off-by: guybd <guy.boudoukh@intel.com>
Signed-off-by: Guy Boudoukh <guy.boudoukh@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
|
2026-07-13 04:16:18 +00:00 |
|
|
zzt
|
2595d5cebc
|
[Model] Optimize Qwen3.5 on H20 (#48350)
Signed-off-by: zzt <zengzetang.zzt@antgroup.com>
|
2026-07-13 03:30:48 +00:00 |
|
|
Tanish MalekarandClaude Sonnet 4.6
|
e26264f3ef
|
[Kernel] Implement CUDA kernel for ReLUSquaredActivation (relu^2) (#39058)
Signed-off-by: Tanish Malekar <tanishmalekar32@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-07-12 19:18:03 -07:00 |
|
|
Canlin Guoandyewentao256
|
a02984ed47
|
[Perf][Qwen] Replace MOE all-reduce with reduce-scatter (#47006)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-12 06:14:49 +00:00 |
|
|
|
19069bcbd5
|
FP32 router GEMV optimization (#48335)
Signed-off-by: peiyuanz <peiyuanz@inferact.ai>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: peiyuanz <peiyuanz@inferact.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: zhouzhou <zhouzhou@zhouzhoudeMacBook-Pro.local>
|
2026-07-11 13:07:48 +00:00 |
|
|
Qiming ZhangandKunshang Ji
|
0b6636cbcb
|
[XPU]remove is_xxx from moe class and bump up kernels (#48079)
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-11 09:27:13 +00:00 |
|
|
Harry Mellor
|
4a6440acef
|
Bump Transformers version to 5.13.0 (#47867)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-11 00:56:14 -07:00 |
|
|
Jimmy Lee
|
092387963c
|
[BugFix] weights processing peak memory reduction for nvfp4 MoE layers (#46276)
Signed-off-by: Jimmy Lee <hirejimmylee@gmail.com>
|
2026-07-11 02:05:35 +00:00 |
|
|
Joe Rowellandmgoin
|
1bf3997eae
|
[Quantization] Bound peak memory when repacking FP4 MoE weights for Marlin (#47851)
Signed-off-by: Joe Rowell <joerowell4@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-10 19:46:13 -06:00 |
|
|
Michael GoinandClaude Fable 5
|
29fd688892
|
Add VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS and skip CuTeDSL fp4_gemm autotuning by default (#48268)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-07-10 18:13:08 -07:00 |
|
|
Michael Goin
|
08dfd68610
|
[Model] Add LongCat-Flash-Lite (n-gram embedding) (#47857)
Signed-off-by: mgoin <mgoin64@gmail.com>
|
2026-07-10 07:17:50 -07:00 |
|
|
|
85c09e9885
|
fix: correct load_weights track logic and enable weight integrity for… (#41811)
Signed-off-by: Yipeng Hu <i26268@metax-tech.com>
Signed-off-by: HuYiPeng <144002351+MynameFelix@users.noreply.github.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Yipeng Hu <i26268@metax-tech.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-07-10 14:08:20 +00:00 |
|
|
FAN YUCHEN
|
fabec87f63
|
[Model] Migrate MistralLarge3ForCausalLM to AutoWeightsLoader (#48153)
Signed-off-by: Yuchen Fan <functionhx@gmail.com>
|
2026-07-10 12:27:58 +00:00 |
|
|
Nick HillandClaude Opus 4.8
|
7614b88ebd
|
[Bugfix][Spec Decode] Fix DFlash draft/target layer-count mismatch (#48113)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-10 04:42:53 -07:00 |
|
|
Jiangyun Zhu
|
300e33797f
|
[Perf] fuse more rmsnorm and all-reduce in qwen3.5 (#46998)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-07-10 15:37:51 +08:00 |
|
|
Yan XuandJingyi Yang
|
95ed0feaa5
|
DCP supports hybrid attention (#40996)
Signed-off-by: YanXu <yancey.yx@alibaba-inc.com>
Signed-off-by: Jingyi Yang <girasoleyang@gmail.com>
Co-authored-by: Jingyi Yang <girasoleyang@gmail.com>
|
2026-07-09 21:34:45 -07:00 |
|
|
gnovack
|
f1a5adddb8
|
update marlin M size for EP (#48144)
Signed-off-by: gnovack <novackgm@gmail.com>
|
2026-07-09 23:52:52 +00:00 |
|
|
ap9272
|
cac3e70cd4
|
Correct model layer aliasing for Bert style models (#43896)
|
2026-07-09 19:46:22 -04:00 |
|
|
|
85b3a7264b
|
[Bugfix][Model Runner V2] Order uniform decodes first so spec decodes aren't misclassified as prefills (#47381)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-09 14:26:27 +01:00 |
|
|
Harry Mellor
|
b83be00cdd
|
Migrate Olmo and Olmo2 to the Transformers modeling backend (#48100)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-09 05:00:23 -07:00 |
|
|
|
412414d8e0
|
Remove PersimmonForCausalLM and FuyuForCausalLM model architectures (#48096)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-09 04:59:08 -07:00 |
|
|
|
ab7961a14a
|
Remove TeleChatForCausalLM (#47989)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-09 00:34:54 -07:00 |
|
|
JooHo LeeandJooHo Lee
|
a07765c6bd
|
[Bugfix] Fix Qwen3-ASR transcription streaming postprocessing (#42478)
Signed-off-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
Signed-off-by: JooHo Lee <jooho414@gmail.com>
Co-authored-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
|
2026-07-09 00:33:27 -07:00 |
|
|
|
bc44f9feb7
|
[ROCm][CI][MoE] Fix double-transpose of fused w3 expert weights (#47874)
Signed-off-by: stefankoncarevic <stefan.koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-08 16:59:28 -07:00 |
|
|
Harry Mellor
|
56da398dac
|
Fix embed scaling + CUDA graphs in Transformers modelling backend (#48010)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-09 00:14:33 +01:00 |
|
|
Martin HickeyandHarry Mellor
|
6cf7b26bd4
|
[docs] Fix the docs build (#48008)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-08 15:47:22 -07:00 |
|
|
Thien Tran
|
0d2f4e7c9c
|
Allow FlashInfer A2A backends for TRTLLM FP8 MoE Modular (#46661)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-07-08 14:58:39 -07:00 |
|
|
Kaihang Jiang
|
089e412878
|
[Perf] Integrate TRTLLM BF16 MoE Modular Kernel (#45182)
Signed-off-by: Kaihang Jiang <kaihangj@login-lyris02.lyris.clusters.nvidia.com>
|
2026-07-09 01:36:14 +04:00 |
|
|
Martin VitandMatthew Bonanni
|
c2ecd0f888
|
Fix FlashAttention MLA prefill V unpadding (#42642)
Signed-off-by: Martin Vit <martin@voipmonitor.org>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-08 15:22:20 +00:00 |
|
|
Michael GoinandClaude Opus 4.8
|
0d12618e98
|
[Spec Decode] Support hybrid (SWA + full attention) DFlash drafters (#47914)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-08 11:12:45 -04:00 |
|
|
|
572b25b03e
|
[Bug] Fix Batched DeepGEMM (#47884)
Signed-off-by: Robert Shaw <robertgshaw2-redhat@dgx-b200-02.mgmt.accl-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2-redhat@dgx-b200-02.mgmt.accl-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-08 09:05:03 -04:00 |
|
|
rasmith
|
934eeaecfb
|
[CI/Build][BugFix][The Rock] Fix get_ssm_device_name to return sanitized, usable filename (#47781)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
|
2026-07-08 12:12:54 +00:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
db39d60010
|
Add tuned selective_state_update float32 config for AMD Instinct MI355 (#47943)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-08 11:41:26 +00:00 |
|
|
|
a1ab51afb6
|
[Bugfix] Allocate HY V3 expert_bias in float32 to prevent silent downcasting (#47797)
Signed-off-by: 辰言 <oncwnuIWp30GguOyJ615Fqj8H-yc@git.weixin.qq.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: 辰言 <oncwnuIWp30GguOyJ615Fqj8H-yc@git.weixin.qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-07-08 11:25:53 +00:00 |
|
|
Thien Tran
|
e7b3853bac
|
Remove router weight upcast for DSv2-related models (#47970)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-07-08 11:19:10 +00:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
eeaf23107f
|
[ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI300X (#47947)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-08 11:09:02 +00:00 |
|
|
Canlin Guo
|
285c08c036
|
[Model] Support MOSS-Transcribe-Diarize (#47729)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
|
2026-07-08 04:05:45 -07:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
1f4ad059d1
|
[ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI300X (#47945)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-08 11:03:58 +00:00 |
|
|
manayangandmanayang
|
51e5372f3d
|
[Model][HunyuanVL] Use native transformers processor and adapt to transformers 5.13 (#47872)
Co-authored-by: manayang <manayang@tencent.com>
|
2026-07-08 07:58:23 +00:00 |
|
|
Hongxia Yang
|
2c64b4c1cc
|
[ROCm] fixed aiter master flag and expert parallelism compatibility on minimax-m3-mxfp8 (#47158)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
|
2026-07-08 15:26:17 +08:00 |
|
|
Wei ZhaoandYongye Zhu
|
2afa3f7e95
|
[Perf] Minimax M3 - Support cross-layer allreduce-norm fusion (#47631)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-07-07 21:16:32 -07:00 |
|
|
Ethan Yangandamd-ethany
|
d9e57ea82e
|
[ROCm][Perf] MXFP8 dense-linear + grouped-MoE GEMM optimizations for MiniMax-M3 (#46117)
Signed-off-by: amd-ethany <amd-ethany@users.noreply.github.com>
Co-authored-by: amd-ethany <amd-ethany@users.noreply.github.com>
|
2026-07-08 04:03:34 +00:00 |
|
|
yzong-rh
|
7d2ce5750e
|
[Bugfix] Patch Hopper MXFP4 OOB scales reads leading to NaN (#47910)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-07-07 22:51:22 +00:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
c8c2f838e7
|
Add tuned selective_state_update config for AMD Instinct MI355 (#47767)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-07 22:19:08 +00:00 |
|
|
Roberto L. Castro
|
c3284c31f5
|
[Perf][3/N] Expand Triton kernel warmup coverage, Qwen (#47546)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
|
2026-07-07 17:06:59 +00:00 |
|
|
Harry Mellor
|
dd94484577
|
Bump Transformers version to 5.10.4 (#41359)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-07 05:13:28 -07:00 |
|