Lyu Han
22013f74ff
Interns2mobius support ( #51149 )
...
Signed-off-by: Lyu Han <lvhan_028@163.com >
2026-08-06 10:39:35 +00:00
anhtra3889 and Khang Ngo
872fd5973e
[Bugfix][LoRA] Guard TrtLlm BF16 MoE LoRA gate on activation type ( #51002 )
...
Signed-off-by: Anh Tran <anhtra@nvidia.com >
Co-authored-by: Khang Ngo <khngo@nvidia.com >
2026-08-06 18:12:54 +08:00
wang.yuqi and Cyrus Leung
2fa490470d
[Model] Fused mm preprocess normalisation on the Device ( #50411 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-06 03:11:14 -07:00
Matej Sirovatka
9c22668436
[Quantization] Preserve precision in online NVFP4 expert packing ( #50029 )
...
Signed-off-by: S1ro1 <matej.sirovatka@gmail.com >
2026-08-06 02:23:54 -07:00
zofia and opencode
febea17f6a
[Model] Fix weight prefix mapping for native Qwen3.5 text-only checkp… ( #50355 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <opencode@anomaly.co >
2026-08-06 07:40:44 +00:00
470297c143
[HPC Attention Backend] hpc attention backend support bf16 kv cache with fp8 weight ( #50980 )
...
Signed-off-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: VAthree <56064364+VAthree@users.noreply.github.com >
2026-08-06 15:40:14 +08:00
zofia and Kunshang Ji
8060260e1d
[Linear] [Kernel] add block-wise scaled_mm ( #49932 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 14:40:26 +08:00
Gabriel Wu and Yongye Zhu
2e35c529b8
[Bugfix][Quantization] Fix MXFP4 conversion for FlashInfer CUTLASS ( #51038 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-08-05 23:19:22 -07:00
b976e980ef
[XPU] Route AWQ linear through choose_mp_linear_kernel ( #50840 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: opencode <noreply@opencode.ai >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-06 13:50:11 +08:00
d779835a11
[XPU] Support MXFP8 linear weights for INC DeepSeek V4 model ( #48476 )
...
Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com >
Signed-off-by: Xiaochang Wu <xiaochang.wu@intel.com >
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-08-06 13:17:48 +08:00
Qiuyang Yue
777b01d1a8
[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference ( #45254 )
...
Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com >
2026-08-06 05:04:19 +00:00
Li Juqi and Cyrus Leung
821717118f
[BugFix][Pooling] Skip weight-prefix probe when model has WeightsMapper ( #50890 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:33 -07:00
Yunzhu Lu and Cyrus Leung
f84df12c6a
[Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 ( #48413 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:00 -07:00
Thien Tran and OpenAI Codex
9f3169960a
[MoE] Align TRTLLM MXFP4 autotune buckets ( #50942 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-05 18:17:53 -07:00
7c77868cdf
[Bugfix] Size and iterate w13 by shard count for non-gated MoE ( #51125 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Matej Sirovatka <matej.sirovatka@gmail.com >
Co-authored-by: Rishi Puri <riship@nvidia.com >
Co-authored-by: Hsiao-Yuan Chen <littlecircle0730@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 17:24:47 -07:00
Tianyu Guo
b30291cf0c
[EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU ( #50390 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 15:59:54 -07:00
bnellnm
373fe8b83e
[MoE Refactor] Remove MoE legacy code ( #51078 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-08-05 14:35:36 -07:00
Netanel Haber
482c613423
[Bugfix][Humming] Preserve ModelOpt FP8 weight dimensions ( #51093 )
...
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
2026-08-05 14:08:55 -07:00
Michael Goin and OpenAI Codex
8f158d0ee2
[MoE] Share apply_moe_activation support metadata ( #44359 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-05 12:05:47 -07:00
fxmarty-amd
bc37fc970e
Revert [Misc] Avoid importing nixl_ep on every vllm serve config ( #50879 ) ( #51176 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
2026-08-05 11:04:40 -07:00
zexplorerhj
d4da0c55af
[Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support ( #51045 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
2026-08-06 00:38:28 +08:00
66b3c0e61f
[Kernel][Model] Optimize FA4 mm_prefix range lookup ( #50294 )
...
Signed-off-by: Juqi Li <2223621784@qq.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:33:49 -07:00
Gabriel Wu and Pavani Majety
08b8613b7b
[Bugfix][Model] Fix MiniMax-M3 NVFP4 inference correctness ( #48929 )
...
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com >
Co-authored-by: Pavani Majety <pmajety@nvidia.com >
2026-08-05 07:25:46 -07:00
cd930c8e78
[Bugfix] Fix MLA kv_b_proj activation dtype with Marlin FP8 ( #38771 )
...
Signed-off-by: Jacob Zhang <jaco8123@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 07:25:41 -07:00
397094da17
Resolve revision to commit_hash once per model load, via huggingface_hub's resolve_revision ( #49990 )
...
Signed-off-by: Wauplin <lucainp@gmail.com >
Signed-off-by: Lucain <lucainp@gmail.com >
Signed-off-by: Lucain Pouget <lucain@huggingface.co >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-05 13:49:56 +00:00
qizixi and Claude Opus 5
4719a9b8f5
[Bugfix][Spec Decode] Fix EAGLE3 DeepSeek draft crash on non-YaRN rope configs ( #51092 )
...
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-05 14:10:52 +08:00
Wei-Cheng (Wayne) Chiu and OpenAI Codex
7794b1e08b
[Bugfix] Skip Qwen3 deepstack buffers without vision ( #49397 )
...
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-05 12:46:23 +08:00
TrainToGPB and Claude
eb3dce9757
[Bugfix][Model] Gemma3n/Gemma4: pad variable-length audio batches ( #50958 )
...
Signed-off-by: TrainToGPB <kaiba0514@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-05 12:46:00 +08:00
c25093a305
[Kernel] Batch invariant NVFP4 MoE using cutlass ( #40372 )
...
Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-08-05 04:31:32 +00:00
aoshen02 and yihengz
6153dbe036
[R3] Unify routed expert shape configuration ( #50940 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: yihengz <yihengz@nvidia.com >
2026-08-04 20:20:41 -07:00
Yejing Lai and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2cb3ff88cc
[BUGFIX][Quant]Fix test_kv_scale_reload failed ( #50405 )
...
Signed-off-by: Lai, Yejing <yejing.lai@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-05 10:50:54 +08:00
05b7876f50
[MoE][Humming] Support SiTU activation for Kimi-K3 ( #50510 )
...
Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-08-04 12:39:18 -07:00
Wentao Ye
d31de3c421
[Kimi K3 Perf] option to shard the shared expert for non mega case, 16.98 GiB memory/GPU saved ( #50912 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-08-04 14:00:44 -04:00
Linkun and lkchen
7cab4368f2
[MM][CG] Support ViT full CUDA graph for Kimi-K2.5 ( #50929 )
...
Signed-off-by: lkchen <anthropic@lkchen.net >
Co-authored-by: lkchen <anthropic@lkchen.net >
2026-08-04 17:15:56 +00:00
amitz-nv and OpenAI Codex
52c0e3cb08
[Kernel] Add support for Flashinfer Mamba SSU algorithm selection ( #50157 )
...
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-04 09:40:23 -07:00
aoshen02 and aoshen02
7153fd70f6
[Bugfix][MoE] Filter packed expert weights during EP loading ( #49558 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
2026-08-04 16:32:03 +00:00
166f4e2dc3
fix: fuse weightless RMSNorms at their declared width ( #50867 )
...
Signed-off-by: Anuj Bolewar <anujbolewar@gmail.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Anuj Bolewar <anujbolewar@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-04 16:03:09 +00:00
Jiangyun Zhu and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f9c74b4b9c
[Mamba] enable prefix cache by default ( #50991 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-04 15:20:51 +00:00
Nicolò Lucchesi
7b50d2c0bc
[Misc] Avoid importing nixl_ep on every vllm serve config ( #50879 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-04 07:56:54 -07:00
199644d410
[Bugfix][Attention] Guard sparse MLA masked MHA workspace ( #50906 )
...
Signed-off-by: yimdev <5779256+yimdev@users.noreply.github.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
2026-08-04 09:59:33 -04:00
Michał Ganczarenko
7c40d61eaf
[Bugfix] Flatten >2D multimodal embeddings, not just 3D ( #50250 )
...
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com >
2026-08-04 13:34:58 +00:00
Andrii Skliar and Andrii Skliar
5789897aa4
[Spec Decode] Add top-k DSpark Markov projection ( #49969 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-08-04 12:52:53 +00:00
Harry Mellor and Claude Opus 5
1eb3694521
[Docs] Fix two docs build warnings ( #51014 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-04 05:29:04 -07:00
Harry Mellor
59b2fdfc4e
Support MLA properly in the Transformers modeling backend ( #48250 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-08-04 13:24:11 +01:00
Lazurite
6a9fdf0dc7
[Bugfix] Resolve seq-cls num_labels from the top-level config for multimodal checkpoints ( #50950 )
...
Signed-off-by: Lazurite <43494437+Rapisurazurite@users.noreply.github.com >
2026-08-04 11:39:39 +00:00
fxmarty-amd and Andreas Karatzas
385d4c084e
[ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing ( #50859 )
...
Signed-off-by: Felix Marty <Felix.Marty@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-03 21:07:46 -05:00
zzt
413e70d52c
Perf/h20 moe config e256 n512 ( #48825 )
...
Signed-off-by: zzt <zengzetang.zzt@antgroup.com >
2026-08-03 17:47:58 -07:00
c810937573
[Kernel] Extend CuTe DSL skinny GEMM to GLM-5.2 ( #49791 )
...
Signed-off-by: Peiyuan Zhou <peiyuanzhou1994@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-03 16:49:41 -07:00
Ryan Hamby and Isotr0py
6a9109d865
[Bugfix] Fix Qwen3-Omni crash on video with no audio track when use_audio_in_video=True ( #48420 )
...
Signed-off-by: RyanJHamby <ryanhamby22@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-08-03 16:22:00 -07:00
0b37d8389f
fix: NVFP4 quantization out_dtype should match model dtype, not torch default ( #48861 )
...
Signed-off-by: Chris Fontes <2224082+fattchris@users.noreply.github.com >
Co-authored-by: Chris Fontes <chris@fontes.io >
Co-authored-by: Chris Fontes <chris@fontes.dev >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-03 22:47:37 +00:00
Lyu Han