b976e980ef
[XPU] Route AWQ linear through choose_mp_linear_kernel ( #50840 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <noreply@opencode.ai >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:50:11 +08:00
d779835a11
[XPU] Support MXFP8 linear weights for INC DeepSeek V4 model ( #48476 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 13:17:48 +08:00
Qiuyang Yue
777b01d1a8
[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference ( #45254 )
...
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com >
2026-08-06 05:04:19 +00:00
Li Juqi and Cyrus Leung
821717118f
[BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper ( #50890 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:33 -07:00
Yunzhu Lu and Cyrus Leung
f84df12c6a
[Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 ( #48413 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:00 -07:00
Thien Tran and OpenAI Codex
9f3169960a
[MoE] Align TRTLLM MXFP4 autotune buckets ( #50942 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-05 18:17:53 -07:00
7c77868cdf
[Bugfix] Size and iterate w13 by shard count for non-gated MoE ( #51125 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 17:24:47 -07:00
Tianyu Guo
b30291cf0c
[EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU ( #50390 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 15:59:54 -07:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Netanel Haber
482c613423
[Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions ( #51093 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-08-05 14:08:55 -07:00
Michael Goin and OpenAI Codex
8f158d0ee2
[MoE] Share apply_moe_activation support metadata ( #44359 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-05 12:05:47 -07:00
fxmarty-amd
bc37fc970e
Revert [Misc] Avoid importing nixl_ep on every vllm serve config ( #50879 ) ( #51176 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-05 11:04:40 -07:00
zexplorerhj
d4da0c55af
[Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support ( #51045 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
2026-08-06 00:38:28 +08:00
66b3c0e61f
[Kernel][Model] Optimize FA4 mm_prefix range lookup ( #50294 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:33:49 -07:00
Gabriel Wu and Pavani Majety
08b8613b7b
[Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness ( #48929 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Pavani Majety <pmajety@nvidia.com >
2026-08-05 07:25:46 -07:00
cd930c8e78
[Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 ( #38771 )
...
Signed-off-by: Jacob Zhang <jaco8123@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:25:41 -07:00
397094da17
Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision ( #49990 )
...
Signed-off-by: Wauplin <lucainp@gmail.com >
Signed-off-by: Lucain <lucainp@gmail.com >
Signed-off-by: Lucain Pouget <lucain@huggingface.co >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-05 13:49:56 +00:00
qizixi and Claude Opus 5
4719a9b8f5
[Bugfix][Spec Decode] Fix EAGLE3 DeepSeek draft crash on non-YaRN rope configs ( #51092 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 14:10:52 +08:00
Wei-Cheng (Wayne) Chiu and OpenAI Codex
7794b1e08b
[Bugfix] Skip Qwen3 deepstack buffers without vision ( #49397 )
...
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-05 12:46:23 +08:00
TrainToGPB and Claude
eb3dce9757
[Bugfix][Model] Gemma3n/Gemma4: pad variable-length audio batches ( #50958 )
...
Signed-off-by: TrainToGPB <kaiba0514@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-05 12:46:00 +08:00
c25093a305
[Kernel] Batch invariant NVFP4 MoE using cutlass ( #40372 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-08-05 04:31:32 +00:00
aoshen02 and yihengz
6153dbe036
[R3] Unify routed expert shape configuration ( #50940 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: yihengz <yihengz@nvidia.com >
2026-08-04 20:20:41 -07:00
Yejing Lai and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2cb3ff88cc
[BUGFIX][Quant]Fix test_kv_scale_reload failed ( #50405 )
...
Signed-off-by: Lai, Yejing <yejing.lai@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 10:50:54 +08:00
05b7876f50
[MoE][Humming] Support SiTU activation for Kimi-K3 ( #50510 )
...
Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-08-04 12:39:18 -07:00
Wentao Ye
d31de3c421
[Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved ( #50912 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-04 14:00:44 -04:00
Linkun and lkchen
7cab4368f2
[MM][CG] Support ViT full CUDA graph for Kimi-K2.5 ( #50929 )
...
Signed-off-by: lkchen <anthropic@lkchen.net >
Co-authored-by: lkchen <anthropic@lkchen.net >
2026-08-04 17:15:56 +00:00
amitz-nv and OpenAI Codex
52c0e3cb08
[Kernel] Add support for Flashinfer Mamba SSU algorithm selection ( #50157 )
...
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-04 09:40:23 -07:00
aoshen02 and aoshen02
7153fd70f6
[Bugfix][MoE] Filter packed expert weights during EP loading ( #49558 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-08-04 16:32:03 +00:00
166f4e2dc3
fix: fuse weightless RMSNorms at their declared width ( #50867 )
...
Signed-off-by: Anuj Bolewar <anujbolewar@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Anuj Bolewar <anujbolewar@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-04 16:03:09 +00:00
Jiangyun Zhu and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f9c74b4b9c
[Mamba] enable prefix cache by default ( #50991 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 15:20:51 +00:00
Nicolò Lucchesi
7b50d2c0bc
[Misc] Avoid importing nixl_ep on every vllm serve config ( #50879 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-04 07:56:54 -07:00
199644d410
[Bugfix][Attention] Guard sparse MLA masked MHA workspace ( #50906 )
...
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-08-04 09:59:33 -04:00
Michał Ganczarenko
7c40d61eaf
[Bugfix] Flatten >2D multimodal embeddings, not just 3D ( #50250 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
2026-08-04 13:34:58 +00:00
Andrii Skliar and Andrii Skliar
5789897aa4
[Spec Decode] Add top-k DSpark Markov projection ( #49969 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-08-04 12:52:53 +00:00
Harry Mellor and Claude Opus 5
1eb3694521
[Docs] Fix two docs build warnings ( #51014 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-04 05:29:04 -07:00
Harry Mellor
59b2fdfc4e
Support MLA properly in the Transformers modeling backend ( #48250 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-04 13:24:11 +01:00
Lazurite
6a9fdf0dc7
[Bugfix] Resolve seq-cls num_labels from the top-level config for multimodal checkpoints ( #50950 )
...
Signed-off-by: Lazurite <43494437+Rapisurazurite@users.noreply.github.com >
2026-08-04 11:39:39 +00:00
fxmarty-amd and Andreas Karatzas
385d4c084e
[ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing ( #50859 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-03 21:07:46 -05:00
zzt
413e70d52c
Perf/h20 moe config e256 n512 ( #48825 )
...
Signed-off-by: zzt <zengzetang.zzt@antgroup.com >
2026-08-03 17:47:58 -07:00
c810937573
[Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 ( #49791 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-03 16:49:41 -07:00
Ryan Hamby and Isotr0py
6a9109d865
[Bugfix] Fix Qwen3-Omni crash on video with no audio track when use_audio_in_video=True ( #48420 )
...
Signed-off-by: RyanJHamby <ryanhamby22@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-08-03 16:22:00 -07:00
0b37d8389f
fix: NVFP4 quantization out_dtype should match model dtype, not torch default ( #48861 )
...
Signed-off-by: Chris Fontes <2224082+fattchris@users.noreply.github.com >
Co-authored-by: Chris Fontes <chris@fontes.io >
Co-authored-by: Chris Fontes <chris@fontes.dev >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-03 22:47:37 +00:00
skysnow2001 and Douglas Lehr
f43e1d26e3
[ROCm] Enable AITER and FP8 inference on GFX120x ( #43615 )
...
Signed-off-by: skysnow2001 <skysnow9285@gmail.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-08-03 16:45:37 -05:00
Raphaël Rialland
4a3447d200
[Bugfix][Model Runner V2] Restore multimodal draft capability detection ( #50417 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
2026-08-03 14:18:14 -07:00
aoshen02 and Nick Hill
42ab184ea7
[MRV2] Enable routed-experts capture ( #50721 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-03 21:00:13 +00:00
Taneem Ibrahim and Wentao Ye
1c0d207915
[Bugfix] Default Gemma3 Model intermediate_tensors to None ( #50777 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-08-03 15:07:59 -04:00
Hongxia Yang
8f50685c48
[ROCm][Kimi-K3] aiter moe environment variable cleanup ( #50582 )
...
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com >
2026-08-03 18:54:04 +00:00
wangxiyuan
dd11df04f3
[Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation ( #49389 )
...
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com >
2026-08-03 11:15:13 -04:00
Wentao Ye
0cf49a5d15
[Refactor] Remove multiple dead codes ( #50285 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-03 10:59:08 -04:00
Kyungmin Lee and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9ae11a6b89
[Model] Add K-EXAONE-2.0-750B-A37B ( #50524 )
...
Signed-off-by: lkm2835 <lkm2835@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-03 06:33:02 -07:00
