CynicDora
256dbcaabf
[Feature] Support custom callable proposer backend for speculative decoding ( #39487 )
...
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn >
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn >
2026-05-13 16:53:01 +00:00
Wentao Ye
e35c0d4c63
[Feature] Support compile mode for batch invariance on SM80 ( #42456 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-05-13 11:02:39 -04:00
Ronen Schaffer
11f6b545d4
[kv_offload] Add multi-tier KV cache offloading framework ( #40020 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
2026-05-13 17:21:43 +03:00
TJian and Yongye Zhu
a8887c208f
[Bugfix] [ROCm] [DSV4] [Perf] Add aiter mhc support ( #41946 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-05-13 21:43:15 +08:00
Marek Wawrzos
67671692ac
[CI] Re-enable Nemotron Parse parity test and switch testing to nemotron-parse v1.2 ( #42498 )
...
Signed-off-by: <mwawrzos@nvidia.com >
2026-05-13 21:05:27 +08:00
hissu-hyvarinen
0a62f5eec9
[AMD] skip machete tests for rocm ( #42326 )
...
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com >
2026-05-13 12:11:03 +00:00
Artur Fierka and quinnlp
3c413a5481
Triton attention: add USE_TD constexpr for tensor descriptor Q/K/V load/store ( #40327 )
...
Signed-off-by: Artur Fierka <artur.fierka@intel.com >
Co-authored-by: quinnlp <quinnlp@users.noreply.github.com >
2026-05-13 13:57:41 +02:00
Ronen Schaffer
79fd1bc7ed
[kv_offload] Add req_id to ReqContext for per-request tracking ( #42507 )
...
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com >
2026-05-13 11:11:10 +00:00
JooHo Lee and JooHo Lee
16863072ca
[Bugfix] Fix scipy audio resampling ratio ( #42233 )
...
Signed-off-by: JooHo Lee <BWAAEEEK@users.noreply.github.com >
Co-authored-by: JooHo Lee <BWAAEEEK@users.noreply.github.com >
2026-05-13 18:52:41 +08:00
hallerite and Claude Opus 4.7
97c4317bf5
[Bugfix][Frontend] Default max_tokens server-side on /inference/v1/generate ( #42329 )
...
Signed-off-by: hallerite <git@hallerite.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-13 11:16:46 +02:00
Yifan Qiao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
13bf242100
[Feat][KVConnector] Add bind_gpu_block_pool() to KVConnectorBase_V1 ( #39654 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-13 02:10:29 -07:00
Yifan Qiao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9ce74042d3
[Bugfix][SimpleCPUOffloadBackend] Dedup in-flight CPU offload stores across scheduler steps ( #41289 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-13 01:53:32 -07:00
Shanshan Shen
92def124bc
[MM][Perf][CG] Support ViT full CUDA graph for Qwen3.5 ( #42151 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-05-13 16:00:32 +08:00
Nicolò Lucchesi
71bcd02ef3
[Bugfix][PD] Fix multi-node TP (TP>8) ( #39907 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-12 22:20:57 -07:00
bnellnm
18f6bf5a21
[MoE Refactor] Add sequence parallel tests to test_moe_layer.py ( #41299 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-05-12 21:52:19 -04:00
+3
ebeb09d822
[KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store ( #40900 )
...
Signed-off-by: leichao.lc <leichao.lc@antgroup.com >
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: leichao.lc <leichao.lc@antgroup.com >
Co-authored-by: ivanium <yifanqiao@inferact.ai >
Co-authored-by: aoshen524 <aoshen@inferact.ai >
Co-authored-by: Dao007forever <daole@inferact.ai >
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com >
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com >
Co-authored-by: foraxe <1055696449@qq.com >
Co-authored-by: Skywalker-EP <173423846@qq.com >
Co-authored-by: fems14 <1804143737@qq.com >
Co-authored-by: jianzs <zheng.shoujian@outlook.com >
Co-authored-by: baxingpiaochong <771405853@qq.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 16:09:10 -07:00
Nick Hill
fe8b42e80c
[CI] Fix test_async_scheduling.py flakiness ( #42455 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-12 21:38:32 +00:00
Giancarlo Delfin
fe5b4e0fe7
[Model Runner V2] Apply synthetic mode to probabilistic rejection sampler ( #41035 )
2026-05-12 13:37:03 -07:00
KaivalyaMDabhadkar
67c89fe40a
[Model][Bugfix] Fix Step3-VL image_embeds input path ( #42333 )
...
Signed-off-by: Kaivalya Dabhadkar <kdabhadkar@nvidia.com >
2026-05-12 18:47:55 +00:00
bnellnm and Robert Shaw
d9b4990783
[MoE Refactor] EPLB refactoring for FusedMoE ( #41055 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-05-12 14:16:31 -04:00
4d591db470
[MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK ( #40735 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 13:37:44 -04:00
yzong-rh
6ff7405b81
[Bugfix] [Frontend] Responses API, fix merging of messages ( #42189 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Signed-off-by: Yifan <yzong@redhat.com >
2026-05-12 16:09:59 +00:00
Yan Ru Pei
bcb9c133ba
feat(kv-events): emit KV cache metadata ( #40984 )
...
Signed-off-by: PeaBrane <yanrpei@gmail.com >
2026-05-12 15:58:48 +00:00
Martin Hickey
418ba8ef14
[kv_offload][BugFix] Fix store deferral ( #41945 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-05-12 18:04:44 +03:00
Avishek Goswami and Avishek Goswami
289cee0473
[vLLM IR] Minor improvements ( #39362 ) ( #39558 )
...
Signed-off-by: Avishek Goswami <avishek.goswami@ibm.com >
Co-authored-by: Avishek Goswami <avishek.goswami@ibm.com >
2026-05-12 10:58:36 -04:00
shanjiaz
6ccb10d794
Added peagle speculators support ( #41826 )
...
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com >
2026-05-12 07:55:57 -07:00
tc-mb and caitianchi
7a9cc5e7f0
[Model] Support MiniCPM-V 4.6 ( #41254 )
...
Signed-off-by: caitianchi <caitianchi@tc-mb.com >
Signed-off-by: tc-mb <157115220+tc-mb@users.noreply.github.com >
Co-authored-by: caitianchi <caitianchi@tc-mb.com >
2026-05-12 14:28:10 +00:00
zhanqiuhu and NickLucche
593d5a4033
[Bugfix] Fix mismatched kernel-per-logical blocks in NIXL HMA transfer ( #42097 )
...
Signed-off-by: ZhanqiuHu <zhu@redhat.com >
Signed-off-by: Zhanqiu Hu <zhu@redhat.com >
Signed-off-by: NickLucche <nlucches@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
2026-05-12 15:53:30 +02:00
bnellnm
6427603ae8
[MoE Refactor] Move remaining experts classes to experts directory ( #42334 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
2026-05-12 09:19:46 -04:00
206eaed08d
[MoE Refactor] Move expert map related code into ExpertMapManager class ( #41046 )
...
Signed-off-by: Bill Nell <bnell@redhat.com >
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
2026-05-12 09:18:27 -04:00
Dipika Sikka
a7b801e26d
[MXFP4] Support for linear layers + compressed-tensors integration ( #41664 )
2026-05-12 07:49:33 -04:00
haosdent
fc8bf6eedb
[CI] De-flake Language Models Test (Extended Generation) test_models(False-False-5-32-bigcode/starcoder2-3b) ( #42392 )
...
Signed-off-by: haosdent <haosdent@gmail.com >
2026-05-12 10:46:48 +00:00
liuzhenwei
07a40ede19
[UT][XPU] fix test_parallel_sampling due to global random state ( #42388 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-05-12 18:03:23 +08:00
Kevin H. Luu
1ff9d33535
[CI] Migrate remaining B200 jobs to b200-k8s with test fixes ( #42387 )
...
Signed-off-by: khluu <khluu000@gmail.com >
2026-05-12 02:00:37 -07:00
chazen and Chauncey
7f65f84428
[Bugfix] Fix empty channel/recipient in harmony for /v1/responses ( #35540 )
...
Signed-off-by: kg6-sleipnir <christopherhazen42@gmail.com >
Signed-off-by: chazen <45186108+kg6-sleipnir@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-05-12 08:45:51 +00:00
wang.yuqi and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
d37e25ffbe
[Frontend] Consolidate Speech to Text entrypoints. ( #42370 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-12 07:06:57 +00:00
ZiTian Zhao
630492da30
[Fix] Gemma4 Mixed-Resolution Image Co-Batching Crash ( #42217 )
...
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com >
2026-05-12 03:13:03 +00:00
d7af6b34d8
[Model Runner V2] Bug fix: logprob dtype int64/int32 issue ( #41761 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-11 21:55:43 +00:00
wang.yuqi
a0dc7a0f36
[CI] Consolidate Speech to Text tests ( #42274 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-11 19:50:17 +00:00
a721315488
[ROCm][Perf] Fix RMSNorm+Quant fusion for gfx950 (non-fnuz) ( #41825 )
...
Signed-off-by: Frida Andersson <fanderss@amd.com >
Signed-off-by: Chuan Li <chuali@amd.com >
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Chuan Li <chuali@amd.com >
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com >
Co-authored-by: Frida Andersson <frida-andersson@users.noreply.github.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
2026-05-11 15:00:51 -04:00
Hexiang Wang
7863fff6e5
[ROCm][DSv4] implement flash sparse mla with triton kernels ( #41812 )
...
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com >
2026-05-11 09:27:11 -07:00
Wentao Ye
0d453e2336
[Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement ( #40408 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-05-11 12:20:58 -04:00
Martin Hickey and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
4955990f1b
[kv_offload] Move FilterReusedOffloadingManager logic to CPUOffloadingManager ( #41727 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-11 18:09:29 +03:00
Thien Tran and Yongye Zhu
724ed2fc35
[DSv4] Improved dequant gather K cache kernel ( #42236 )
...
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
2026-05-11 10:41:12 -04:00
Rohan Potdar and ElizaWszola
a51376b3f0
[Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA ( #40392 )
...
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com >
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com >
Co-authored-by: ElizaWszola <ewszola@redhat.com >
2026-05-11 14:10:50 +00:00
Noa Neria
ac062147fa
Avoid silent weights corruption when loading Nemotron Nano VL with reusable-buffer loaders like runai distributed streaming ( #42244 )
...
Signed-off-by: Noa Neria <nneria@nvidia.com >
2026-05-11 12:03:14 +00:00
Chauncey
617239b70c
[Frontend]Responses API supports chat_template_kwargs ( #42272 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-05-11 11:59:39 +00:00
Nicolò Lucchesi
5672d100ed
[KV Connector][NIXL][Bugfix] Fix NIXL handshake failures not honoring kv_load_failure_policy ( #40364 )
...
When NIXL handshake fails (e.g., due to compatibility hash mismatch
between prefill and decode instances), requests fail with "engine dead"
error instead of gracefully falling back to local recomputation as configured
by kv_load_failure_policy='recompute'.
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-11 09:37:21 +00:00
Nicolò Lucchesi
770e9bd6b3
[Nixl][PD] Lease renewal TTL KV blocks on P ( #41383 )
...
Signed-off-by: NickLucche <nlucches@redhat.com >
2026-05-11 09:27:30 +00:00
shaharmor98 and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f9f770ca0b
fix nixl side-channel host selection ( #41806 )
...
Signed-off-by: Shahar Mor <smor@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-11 07:40:37 +00:00
CynicDora