|
85b3a7264b
|
[Bugfix][Model Runner V2] Order uniform decodes first so spec decodes aren't misclassified as prefills (#47381)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-09 14:26:27 +01:00 |
|
|
Harry Mellor
|
b83be00cdd
|
Migrate Olmo and Olmo2 to the Transformers modeling backend (#48100)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-09 05:00:23 -07:00 |
|
|
|
412414d8e0
|
Remove PersimmonForCausalLM and FuyuForCausalLM model architectures (#48096)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-09 04:59:08 -07:00 |
|
|
|
ab7961a14a
|
Remove TeleChatForCausalLM (#47989)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-09 00:34:54 -07:00 |
|
|
JooHo LeeandJooHo Lee
|
a07765c6bd
|
[Bugfix] Fix Qwen3-ASR transcription streaming postprocessing (#42478)
Signed-off-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
Signed-off-by: JooHo Lee <jooho414@gmail.com>
Co-authored-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
|
2026-07-09 00:33:27 -07:00 |
|
|
|
bc44f9feb7
|
[ROCm][CI][MoE] Fix double-transpose of fused w3 expert weights (#47874)
Signed-off-by: stefankoncarevic <stefan.koncarevic@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-08 16:59:28 -07:00 |
|
|
Harry Mellor
|
56da398dac
|
Fix embed scaling + CUDA graphs in Transformers modelling backend (#48010)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-09 00:14:33 +01:00 |
|
|
Martin HickeyandHarry Mellor
|
6cf7b26bd4
|
[docs] Fix the docs build (#48008)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-08 15:47:22 -07:00 |
|
|
Thien Tran
|
0d2f4e7c9c
|
Allow FlashInfer A2A backends for TRTLLM FP8 MoE Modular (#46661)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-07-08 14:58:39 -07:00 |
|
|
Kaihang Jiang
|
089e412878
|
[Perf] Integrate TRTLLM BF16 MoE Modular Kernel (#45182)
Signed-off-by: Kaihang Jiang <kaihangj@login-lyris02.lyris.clusters.nvidia.com>
|
2026-07-09 01:36:14 +04:00 |
|
|
Martin VitandMatthew Bonanni
|
c2ecd0f888
|
Fix FlashAttention MLA prefill V unpadding (#42642)
Signed-off-by: Martin Vit <martin@voipmonitor.org>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
|
2026-07-08 15:22:20 +00:00 |
|
|
Michael GoinandClaude Opus 4.8
|
0d12618e98
|
[Spec Decode] Support hybrid (SWA + full attention) DFlash drafters (#47914)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-07-08 11:12:45 -04:00 |
|
|
|
572b25b03e
|
[Bug] Fix Batched DeepGEMM (#47884)
Signed-off-by: Robert Shaw <robertgshaw2-redhat@dgx-b200-02.mgmt.accl-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Robert Shaw <robertgshaw2-redhat@dgx-b200-02.mgmt.accl-001.lab.rdu2.dc.redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-08 09:05:03 -04:00 |
|
|
rasmith
|
934eeaecfb
|
[CI/Build][BugFix][The Rock] Fix get_ssm_device_name to return sanitized, usable filename (#47781)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
|
2026-07-08 12:12:54 +00:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
db39d60010
|
Add tuned selective_state_update float32 config for AMD Instinct MI355 (#47943)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-08 11:41:26 +00:00 |
|
|
|
a1ab51afb6
|
[Bugfix] Allocate HY V3 expert_bias in float32 to prevent silent downcasting (#47797)
Signed-off-by: 辰言 <oncwnuIWp30GguOyJ615Fqj8H-yc@git.weixin.qq.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: 辰言 <oncwnuIWp30GguOyJ615Fqj8H-yc@git.weixin.qq.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-07-08 11:25:53 +00:00 |
|
|
Thien Tran
|
e7b3853bac
|
Remove router weight upcast for DSv2-related models (#47970)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2026-07-08 11:19:10 +00:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
eeaf23107f
|
[ROCm] Add tuned selective_state_update float32 config for AMD Instinct MI300X (#47947)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-08 11:09:02 +00:00 |
|
|
Canlin Guo
|
285c08c036
|
[Model] Support MOSS-Transcribe-Diarize (#47729)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
|
2026-07-08 04:05:45 -07:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
1f4ad059d1
|
[ROCm] Add tuned selective_state_update float16 config for AMD Instinct MI300X (#47945)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-08 11:03:58 +00:00 |
|
|
manayangandmanayang
|
51e5372f3d
|
[Model][HunyuanVL] Use native transformers processor and adapt to transformers 5.13 (#47872)
Co-authored-by: manayang <manayang@tencent.com>
|
2026-07-08 07:58:23 +00:00 |
|
|
Hongxia Yang
|
2c64b4c1cc
|
[ROCm] fixed aiter master flag and expert parallelism compatibility on minimax-m3-mxfp8 (#47158)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
|
2026-07-08 15:26:17 +08:00 |
|
|
Wei ZhaoandYongye Zhu
|
2afa3f7e95
|
[Perf] Minimax M3 - Support cross-layer allreduce-norm fusion (#47631)
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
|
2026-07-07 21:16:32 -07:00 |
|
|
Ethan Yangandamd-ethany
|
d9e57ea82e
|
[ROCm][Perf] MXFP8 dense-linear + grouped-MoE GEMM optimizations for MiniMax-M3 (#46117)
Signed-off-by: amd-ethany <amd-ethany@users.noreply.github.com>
Co-authored-by: amd-ethany <amd-ethany@users.noreply.github.com>
|
2026-07-08 04:03:34 +00:00 |
|
|
yzong-rh
|
7d2ce5750e
|
[Bugfix] Patch Hopper MXFP4 OOB scales reads leading to NaN (#47910)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-07-07 22:51:22 +00:00 |
|
|
vanshbhatia-amdandvanshbhatia-amd
|
c8c2f838e7
|
Add tuned selective_state_update config for AMD Instinct MI355 (#47767)
Signed-off-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
Co-authored-by: vanshbhatia-amd <210711135+vanshbhatia-amd@users.noreply.github.com>
|
2026-07-07 22:19:08 +00:00 |
|
|
Roberto L. Castro
|
c3284c31f5
|
[Perf][3/N] Expand Triton kernel warmup coverage, Qwen (#47546)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
|
2026-07-07 17:06:59 +00:00 |
|
|
Harry Mellor
|
dd94484577
|
Bump Transformers version to 5.10.4 (#41359)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-07 05:13:28 -07:00 |
|
|
Shaun Kotek
|
7ff656cc8b
|
fix: ensure no double load of lm head in nemotron mtp (#47440)
Signed-off-by: Shaun Kotek - Nvidia <skotek@nvidia.com>
|
2026-07-07 12:01:45 +00:00 |
|
|
danielafrimi
|
0a2965b1b3
|
[BugFix] Fix ModelOpt mixed-precision quantization for sparse quantized_layers configs. (#47318)
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com>
Signed-off-by: <dafrimi@nvidia.com>
|
2026-07-07 11:45:13 +00:00 |
|
|
Harry Mellor
|
0ed05b6f82
|
[CI] Fix Transformers modeling backend LoRA test (#47832)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-07 11:40:00 +00:00 |
|
|
|
c85d72076a
|
[HARDWARE][POWER] optimize math functions of VSX power (#47321)
Signed-off-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com>
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com>
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com>
Co-authored-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
|
2026-07-07 09:35:47 +00:00 |
|
|
Netanel Haberandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
066f02ae94
|
[MoE] FI autotuning: max bucket = max token count [e.g. DP_size*MNBT] (#47427)
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-07 12:08:36 +03:00 |
|
|
Jee Jee Li
|
5d23ca47ab
|
[Kernel] Applies routed_scaling_factor internally (#47408)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
|
2026-07-07 02:00:54 -07:00 |
|
|
Micah WilliamsonandAndreas Karatzas
|
dd5c299fbe
|
[ROCm][Bugfix] Convert ModelOpt FP8 per-channel weights to e4m3fnuz on MI300/MI325 (#47201)
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
|
2026-07-06 23:24:58 -07:00 |
|
|
Guan-Ming Chiu
|
c64c356990
|
[Perf] Bound DiffusionGemma sampler transient via request-tiled logits (#45672)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
|
2026-07-07 03:42:01 +00:00 |
|
|
Rahul Vishwakarma
|
9dd2465896
|
feat(cpu): add CPU support for Mamba ShortConv (#35059)
Signed-off-by: Rahul Vishwakarma <Rahul.Vishwakarma2@ibm.com>
|
2026-07-07 10:47:08 +08:00 |
|
|
Kyle Sayers
|
445321fab4
|
[Bugfix] [Quantization] Fix loading for CT DSV2 (#47780)
|
2026-07-07 02:28:00 +00:00 |
|
|
Jinzhen Linandmgoin
|
d891b9bd51
|
[Quantization] add humming moe backend to all dense/moe oracles (#41652)
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
|
2026-07-06 13:36:07 -07:00 |
|
|
Harry Mellor
|
ae098abe3f
|
[CI] Fix some errors on main (#47726)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-06 19:40:23 +00:00 |
|
|
Andrii SkliarandAndrii Skliar
|
b1384f5ec6
|
Enable B12x backend for non-gated MoEs (like Nemotron) (#43328)
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
|
2026-07-06 12:40:07 -07:00 |
|
|
Nathaniel McVicarandHarry Mellor
|
b136cc2c2c
|
[Bugfix][Model] Add stability window to DiffusionGemma to match HF stability_threshold semantics (#45965)
Signed-off-by: Nathaniel McVicar <namcvica@microsoft.com>
Signed-off-by: Nathaniel McVicar <Nathaniel.McVicar@microsoft.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-06 19:39:12 +00:00 |
|
|
|
24dd2aec81
|
[Bugfix] Preserve FP8 indexer WK pairs across incremental load_weights (#46168)
Signed-off-by: lcheng <lcheng321@gatech.edu>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-07-06 09:16:46 -07:00 |
|
|
Harry Mellorandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
5bce653e09
|
Make the Transformers modeling backend as fast as native vLLM (#47187)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-06 16:59:14 +01:00 |
|
|
|
5ad11172b7
|
[perf]Add fused Kimi image preprocessing (#47416)
Signed-off-by: Kevin-XiongC <kevin_xiong1997@outlook.com>
Signed-off-by: Kevin_Xiong <kevin_xiong1997@outlook.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-07-06 08:46:32 -07:00 |
|
|
Wentao Ye
|
b1c6dba558
|
[Refactor] Remove multiple dead code (#47329)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-07-06 07:54:08 -07:00 |
|
|
Kunshang JiandHarry Mellor
|
07f9baf756
|
Revert "[Platform] Replace torch.cuda.Event with torch.Event (#47140)" (#47668)
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-06 14:18:33 +01:00 |
|
|
Ayushman SinghandBenjamin Chislett
|
7a90eb98ab
|
[Bugfix] [Gemma4] Fix Gemma4 MTP draft model layers ignoring quant_config (#47091)
Signed-off-by: Ayushman Singh <40520701+ayush1399@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
|
2026-07-06 14:04:00 +01:00 |
|
|
Yan Ma
|
90ce3a09be
|
[bugfix] fix MOSS-Audio deepstack_input_embeds initialization in PP (#47607)
Signed-off-by: Yan Ma <yan.ma@intel.com>
|
2026-07-06 17:15:50 +08:00 |
|
|
Ma JianandKunshang Ji
|
26c754d847
|
[XPU][Bugfix] Do not transpose weight_scale_inv at load time (#47116)
Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
|
2026-07-06 17:15:26 +08:00 |
|