Artem Perevedentsev
333529deae
[EPLB] Fix replica selection bias in fused_moe router ( #40810 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-04-24 22:06:41 +00:00
qli88
095d2f87e8
[Bug] Fix GLM-5.1 running error on ROCm platform ( #40763 )
...
Signed-off-by: Qiang Li <qiang.li2@amd.com >
2026-04-24 19:54:40 +00:00
Alex Brooks
5e11b40365
[Frontend] Delegate to vLLM Omni When --omni Passed ( #40744 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-04-24 12:30:00 -04:00
JartX
914d0464c1
[Refactor] Unify 2D/3D kernels in triton_unified_attention ( #40631 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-04-24 17:18:06 +02:00
Jinzhen Lin
9f771b3ab9
[Quantization] add humming quantization kernel ( #34556 )
2026-04-24 09:29:44 -04:00
Itay Alroy
c9d3c6e6af
fused_moe: treat NIXL EP as batched experts ( #40412 )
...
Signed-off-by: Itay Alroy <ialroy@nvidia.com >
2026-04-24 08:05:31 -05:00
Or Ozeri
51adca74e6
[kv_offload+HMA][9/N]: Support lookup with multiple KV groups ( #39401 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-24 15:32:29 +03:00
Netanel Haber
e8eb0490ce
[Bugfix][MoE] Unpad routed output before shared expert add [ Fixes #35949 ] ( #40794 )
...
Signed-off-by: Netanel Haber <nhaber@nvidia.com >
2026-04-24 11:53:23 +00:00
Jiangyun Zhu
e8ee2a78db
[Attention] use diff kv backend for mimo v2 flash ( #40045 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-04-24 11:25:55 +00:00
2ec18f5df4
[Bugfix][Parser] Fix Mistral tool parser for HF tokenizers ( #39294 )
...
Signed-off-by: thomasmaindron <thomasmaindron@users.noreply.github.com >
Co-authored-by: thomasmaindron <thomasmaindron@users.noreply.github.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-04-24 19:01:56 +08:00
milesial
9ad5abe772
Fix Nano Nemotron VL static image inputs ( #40724 )
...
Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com >
2026-04-24 09:18:55 +00:00
512f522192
[Model] Gemma4: add bidirectional vision attention for sliding layers with window guard ( #40534 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Signed-off-by: Luciano Martins <lucianomartins@google.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-04-24 08:27:46 +00:00
Yuwen Zhou and Kunshang Ji
4c34b2f6fc
[XPU] Enable torch.compile for XPU GDN attention ( #39466 )
...
Signed-off-by: yuwenzho <yuwen.zhou@intel.com >
Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-24 16:26:16 +08:00
Jackmin801 and Claude
079a4cf399
[MoE] Move cutlass moe to fused_moe/experts/ ( #40574 )
...
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-24 06:05:49 +00:00
9744b699ba
[Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. ( #40688 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-04-24 05:37:50 +00:00
Bugen Zhao and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
c662b4359e
[Bugfix] Avoid mutating chat_template_kwargs in HYV3ReasoningParser initialization ( #40713 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-24 13:08:58 +08:00
lyd1992
100c7b65e7
[Platform] Fix RISC-V platform detection (lscpu parsing + non-NUMA meminfo) ( #40427 )
...
Signed-off-by: liuyudong <liuyudong@iscas.ac.cn >
2026-04-24 04:33:05 +00:00
Neil Schemenauer
56bdf85e10
[Feature] Avoid eager import of the "mistral_common" package. ( #40043 )
...
Signed-off-by: Neil Schemenauer <nas@arctrix.com >
2026-04-24 02:49:16 +00:00
Nick Hill
e9f331d72e
[MRV2] Ensure warmup covers prefill path ( #40746 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-24 01:33:26 +00:00
c9bf77df92
[BUG]: fix HF tokenizer concurrent borrow in tool parsers ( #40059 )
...
Signed-off-by: Yifan <yzong@redhat.com >
Co-authored-by: timon0305 <timon0305@outlook.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-04-23 18:20:30 -07:00
Doug Campos
92762edc53
[Bugfix] Treat <tool_call> as implicit reasoning end in Qwen3 parser ( #35687 )
...
Signed-off-by: Doug Campos <qmx@qmx.me >
2026-04-24 09:10:04 +08:00
Benjamin Chislett and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
626daa2076
[Feat] Unified Synthetic Acceptance Rate for V1 and V2 ( #40662 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-24 00:48:08 +00:00
Nick Hill
fe85a92e86
[Core] Avoid seq_lens_cpu GPU->CPU sync ( #40654 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-04-24 00:35:55 +00:00
Sage Moore
62b1bbe470
[EPLB] Remove asyncio infrastructure from Async EPLB ( #40730 )
...
Signed-off-by: Sage Moore <sage@neuralmagic.com >
2026-04-24 00:21:15 +00:00
Hemanth Acharya
fa4b70555b
[ROCm] Cast score correction bias tensor during model construction for DeepSeek/Kimi-K2 ( #39999 )
...
Signed-off-by: Hemanth Acharya <heachary@amd.com >
2026-04-24 09:02:12 +09:00
Jackmin801 and Robert Shaw
447c372ac5
[MoE] Move remaining PrepareAndFinalize to prepare finalize folder ( #39009 )
...
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-04-23 20:00:53 -04:00
Sophie du Couédic and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
ff2c2bd80a
[Docs]Add documentation for bench serve visualization arguments ( #40539 )
...
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-23 15:48:29 -07:00
Matthew Bonanni
cde8d24710
[Spec Decode] Move SpecDecodeBaseProposer out of eagle.py ( #40732 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-04-23 22:28:27 +00:00
bnellnm
4a6dd1c3cc
[Bugfix] Fix DeepSeek V2-Lite Accuracy drop ( #40673 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-23 18:11:37 -04:00
czhu-cohere and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
7ff65b1900
[Bugfix] Fix workspace resize leaking reserved GPU memory ( #39226 )
...
Signed-off-by: root <conway.zhu@cohere.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-23 20:50:05 +00:00
1b1c01de39
[MoE] Move xpu moe to fused_moe/experts/ ( #40568 )
...
Signed-off-by: Jackmin801 <ongjackm@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-23 13:38:10 -04:00
e9ba519f45
[DP][Ray] Pin DP control bundle to same node as first GPU bundle ( #39167 )
...
Signed-off-by: Shahar Mor <smor@nvidia.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-23 17:21:13 +00:00
Or Ozeri
5ef33ab250
[kv_offload+HMA][10/N]: Support load with multiple KV groups ( #39402 )
...
Signed-off-by: Or Ozeri <oro@il.ibm.com >
2026-04-23 20:00:45 +03:00
bnellnm
1c2c1eb8b9
[MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping ( #40671 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-04-23 11:22:34 -04:00
pschlan-amd and TJian
0098db9ec1
[ROCm] Implement GPU-to-NUMA-node detection ( #40015 )
...
Signed-off-by: Patrick Schlangen <pschlan@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-04-23 10:08:48 -05:00
b7a2605020
[Bugfix] Make Attention Backend Auto-Selection Batch-Invariance-Aware ( #40193 )
...
Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-23 14:57:03 +00:00
stevenkuang and Jee Jee Li
d0009ddb0b
[Model] Support Hy3 preview ( #40681 )
...
Signed-off-by: stevenkuang <stevenkuang@tencent.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-23 22:08:26 +08:00
Richard Zou
424033f4fc
[Bugfix] Include inductor and functorch configs in compilation cache key ( #40627 )
...
Signed-off-by: Richard Zou <zou3519@gmail.com >
2026-04-23 09:52:59 -04:00
Isotr0py
da1e7311ca
[Misc] use model arch converter for bidi models identification ( #40701 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-04-23 13:42:52 +00:00
BadrBasowid
2196bac135
[Compilation] Refactor SiluMul activation+quant Fusion Pass ( #39684 )
...
Signed-off-by: BadrBasowid <badr.basowid@gmail.com >
2026-04-23 09:10:36 -04:00
Shengqi Chen and Claude Opus 4.6
3ed5231c6a
[Build] Switch default CUDA to 13.0, update CUDA architecture lists, clean up stale build-args ( #39878 )
...
Signed-off-by: Shengqi Chen <harry-chen@outlook.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-23 15:51:28 +08:00
Nicolò Lucchesi
9c2492e501
[Misc] Support Human-readable (k/K/m/M..) json cli arg ( #40473 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-04-23 09:42:23 +02:00
Shanshan Shen
fe57be7809
[MM][CG] Support --enable-vit-cuda-graph option for VLM examples ( #40580 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-04-22 22:46:14 -07:00
Flora Feng and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8317cedc77
[Responses] Add tool_choice/tools validation to match OpenAI behavior ( #40399 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 22:46:10 -07:00
Zhengxu Chen
98a242ff61
[compile] Skip FX graph deserialiaztion on loading, further reducing warm compile time. ( #40151 )
...
Signed-off-by: zhxchen17 <zhxchen17@fb.com >
2026-04-23 13:43:18 +08:00
e4ee48da2d
[MoE refactor] refactor GPTQMarlinMoEMethod with MK ( #37990 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-04-23 05:21:47 +00:00
Kunshang Ji
342c58bc54
[BugFix]fix Qwen3 MoE call gate twice ( #40664 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-23 05:04:41 +00:00
fe9c3d6c5f
[TurboQuant] enable FA3/FA4 for prefill paths ( #40092 )
...
Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com >
Co-authored-by: 墨楼 <huangzhilin.hzl@antgroup.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: Codex <codex@openai.com >
2026-04-23 07:35:24 +03:00
Chaojun Zhang and Kunshang Ji
ccaf5ffaa3
[XPU] disable fusion pattern support on XPU platform ( #39789 )
...
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-04-23 10:07:45 +08:00
Lucas Kabela
0283f303d8
[BE] Fix compile time message to be consistent (use monitoring) ( #40641 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-04-23 00:12:08 +00:00
Artem Perevedentsev