Commit Graph
11858 Commits
Author SHA1 Message Date
CynicDora 256dbcaabf [Feature] Support custom callable proposer backend for speculative decoding (#39487)
Signed-off-by: 524031910363 <hyzhyzsh@sjtu.edu.cn>
Signed-off-by: CynicDora <hyzhyzsh@sjtu.edu.cn>
2026-05-13 16:53:01 +00:00
Ronen Schaffer 11f6b545d4 [kv_offload] Add multi-tier KV cache offloading framework (#40020)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
2026-05-13 17:21:43 +03:00
TJianandYongye Zhu a8887c208f [Bugfix] [ROCm] [DSV4] [Perf] Add aiter mhc support (#41946)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
2026-05-13 21:43:15 +08:00
zofiaandKunshang Ji 0ddaf6dffa [XPU] [CT] Enable CT W4A4MxFp4 path and add xpu kernel (#38896)
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: zofia <110436990+zufangzhu@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-13 06:43:00 -07:00
PikaPikachu 3b1ef03be4 [Bugfix][Quark] Fix W8A8 INT8 garbage outputs on Step-3.5-Flash (and other 3-key fused-MoE Quark exports) (#41892)
Signed-off-by: kangletian <kangletian@hotmail.com>
2026-05-13 11:59:49 +00:00
Artur Fierkaandquinnlp 3c413a5481 Triton attention: add USE_TD constexpr for tensor descriptor Q/K/V load/store (#40327)
Signed-off-by: Artur Fierka <artur.fierka@intel.com>
Co-authored-by: quinnlp <quinnlp@users.noreply.github.com>
2026-05-13 13:57:41 +02:00
Ronen Schaffer 79fd1bc7ed [kv_offload] Add req_id to ReqContext for per-request tracking (#42507)
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
2026-05-13 11:11:10 +00:00
SILONG ZENG cee6751e54 [Bugfix][Qwen3-VL] Fix pipeline-parallel deepstack initialization (#42394)
Signed-off-by: MrZ20 <2609716663@qq.com>
2026-05-13 10:58:42 +00:00
JooHo LeeandJooHo Lee 16863072ca [Bugfix] Fix scipy audio resampling ratio (#42233)
Signed-off-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
Co-authored-by: JooHo Lee <BWAAEEEK@users.noreply.github.com>
2026-05-13 18:52:41 +08:00
Andreas Karatzas d628a3c5cb [ROCm][CI] Skip ROCm batch invalid-input test pending torch fix (#41572)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
2026-05-13 18:50:47 +08:00
akii96 74dffae666 [ROCm] Run AITER RMSNorm pad fusion before AR RMS fusion (#42411)
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
2026-05-13 18:35:12 +08:00
halleriteandClaude Opus 4.7 97c4317bf5 [Bugfix][Frontend] Default max_tokens server-side on /inference/v1/generate (#42329)
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 11:16:46 +02:00
Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 13bf242100 [Feat][KVConnector] Add bind_gpu_block_pool() to KVConnectorBase_V1 (#39654)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-13 02:10:29 -07:00
Yifan Qiaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 9ce74042d3 [Bugfix][SimpleCPUOffloadBackend] Dedup in-flight CPU offload stores across scheduler steps (#41289)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-13 01:53:32 -07:00
sychen52 a8c13d2837 Patch SlidingWindowSpec.real_page_size_bytes for nvfp4 kv (#42464)
Signed-off-by: Shiyang Chen <shiychen@nvidia.com>
2026-05-13 01:46:30 -07:00
Shanshan Shen 92def124bc [MM][Perf][CG] Support ViT full CUDA graph for Qwen3.5 (#42151)
Signed-off-by: shen-shanshan <467638484@qq.com>
2026-05-13 16:00:32 +08:00
Taneem Ibrahimgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Wentao Ye
503697c9ce [chore] Refactor pooling metadata token ID accessors (#42368)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-05-13 06:08:01 +00:00
Nicolò Lucchesi 71bcd02ef3 [Bugfix][PD] Fix multi-node TP (TP>8) (#39907)
Signed-off-by: NickLucche <nlucches@redhat.com>
2026-05-12 22:20:57 -07:00
bnellnm 18f6bf5a21 [MoE Refactor] Add sequence parallel tests to test_moe_layer.py (#41299)
Signed-off-by: Bill Nell <bnell@redhat.com>
2026-05-12 21:52:19 -04:00
Wentao Ye 3d635c58c0 [Perf] Optimize MLA compute_prefill_context memory allocation (#42460)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-05-12 16:23:46 -07:00
+3 ebeb09d822 [KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store (#40900)
Signed-off-by: leichao.lc <leichao.lc@antgroup.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
Co-authored-by: ivanium <yifanqiao@inferact.ai>
Co-authored-by: aoshen524 <aoshen@inferact.ai>
Co-authored-by: Dao007forever <daole@inferact.ai>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com>
Co-authored-by: foraxe <1055696449@qq.com>
Co-authored-by: Skywalker-EP <173423846@qq.com>
Co-authored-by: fems14 <1804143737@qq.com>
Co-authored-by: jianzs <zheng.shoujian@outlook.com>
Co-authored-by: baxingpiaochong <771405853@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-12 16:09:10 -07:00
Giancarlo Delfin fe5b4e0fe7 [Model Runner V2] Apply synthetic mode to probabilistic rejection sampler (#41035) 2026-05-12 13:37:03 -07:00
0ce6613b9c platforms: add uses_cpu_device() hook to Platform for DeviceConfig (#42313)
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-05-12 12:39:17 -07:00
KaivalyaMDabhadkar 67c89fe40a [Model][Bugfix] Fix Step3-VL image_embeds input path (#42333)
Signed-off-by: Kaivalya Dabhadkar <kdabhadkar@nvidia.com>
2026-05-12 18:47:55 +00:00
bnellnmandRobert Shaw d9b4990783 [MoE Refactor] EPLB refactoring for FusedMoE (#41055)
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-05-12 14:16:31 -04:00
4d591db470 [MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK (#40735)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
2026-05-12 13:37:44 -04:00
yzong-rh 6ff7405b81 [Bugfix] [Frontend] Responses API, fix merging of messages (#42189)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Yifan <yzong@redhat.com>
2026-05-12 16:09:59 +00:00
Yan Ru Pei bcb9c133ba feat(kv-events): emit KV cache metadata (#40984)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
2026-05-12 15:58:48 +00:00
Jonathan MamouandWentao Ye c8a6e272e0 [CPU] Fix rotary embedding for CPU without flash-attn ops (#42225)
Signed-off-by: jmamou <jonathan.mamou@intel.com>
Signed-off-by: Jonathan Mamou <jonathan.mamou@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-05-12 15:05:35 +00:00
Wentao Ye a1b2d87498 [Refactor] Clean up pooling models build_tok_params logic (#42341)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
2026-05-12 15:05:05 +00:00
Martin Hickey 418ba8ef14 [kv_offload][BugFix] Fix store deferral (#41945)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-05-12 18:04:44 +03:00
Avishek GoswamiandAvishek Goswami 289cee0473 [vLLM IR] Minor improvements (#39362) (#39558)
Signed-off-by: Avishek Goswami <avishek.goswami@ibm.com>
Co-authored-by: Avishek Goswami <avishek.goswami@ibm.com>
2026-05-12 10:58:36 -04:00
shanjiaz 6ccb10d794 Added peagle speculators support (#41826)
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
2026-05-12 07:55:57 -07:00
tc-mbandcaitianchi 7a9cc5e7f0 [Model] Support MiniCPM-V 4.6 (#41254)
Signed-off-by: caitianchi <caitianchi@tc-mb.com>
Signed-off-by: tc-mb <157115220+tc-mb@users.noreply.github.com>
Co-authored-by: caitianchi <caitianchi@tc-mb.com>
2026-05-12 14:28:10 +00:00
zhanqiuhuandNickLucche 593d5a4033 [Bugfix] Fix mismatched kernel-per-logical blocks in NIXL HMA transfer (#42097)
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
2026-05-12 15:53:30 +02:00
bnellnm 6427603ae8 [MoE Refactor] Move remaining experts classes to experts directory (#42334)
Signed-off-by: Bill Nell <bnell@redhat.com>
2026-05-12 09:19:46 -04:00
206eaed08d [MoE Refactor] Move expert map related code into ExpertMapManager class (#41046)
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
2026-05-12 09:18:27 -04:00
Thomas ParnellandClaude Opus 4.6 8f89381fc6 [Hybrid] Warmup Mamba2 SSD kernel (#39822)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-12 12:46:22 +00:00
Dipika Sikka a7b801e26d [MXFP4] Support for linear layers + compressed-tensors integration (#41664) 2026-05-12 07:49:33 -04:00
bc03f280c8 [XPU] keep generator state of sycl kernel align with pytorch (#41771)
Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Qiming Zhang <qiming1.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-12 11:44:47 +00:00
chazenandChauncey 7f65f84428 [Bugfix] Fix empty channel/recipient in harmony for /v1/responses (#35540)
Signed-off-by: kg6-sleipnir <christopherhazen42@gmail.com>
Signed-off-by: chazen <45186108+kg6-sleipnir@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-05-12 08:45:51 +00:00
amitz-nv ef34592a1a [Bugfix] Fix double reduce in flashinfer_nvlink_two_sided and flashinfer_nvlink_one_sided backends (#41382)
Signed-off-by: amitz-nv <203509407+amitz-nv@users.noreply.github.com>
2026-05-12 07:47:47 +00:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d37e25ffbe [Frontend] Consolidate Speech to Text entrypoints. (#42370)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-12 07:06:57 +00:00
liuzhenweiandKunshang Ji 8517cdaf90 [XPU] update dp rank w/o env-var isolation (#39856)
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
2026-05-12 14:49:54 +08:00
Lucas Kabela 4e498b5e5c [Bugfix][Performance Improvement] Improve penalties triton kernel performance (#40657)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
2026-05-12 05:47:20 +00:00
Yasmin Moslemandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 28ee78af54 Implement custom dataset class for ASR benchmarking (#41576)
Signed-off-by: Yasmin Moslem <48152713+ymoslem@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-12 12:17:58 +08:00
ZiTian Zhao 630492da30 [Fix] Gemma4 Mixed-Resolution Image Co-Batching Crash (#42217)
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
2026-05-12 03:13:03 +00:00
Chauncey 920bf3ec84 [Bugifx] [Qwen3CoderTool] Restore supports_required_and_named for required tool_choice (#42292)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-05-12 02:09:56 +00:00
pschlan-amd 39dff5ff39 Add VLLM_USE_SPINLOOP_EXT to use more efficient busy polling (#36517)
Signed-off-by: Patrick Schlangen <pschlan@amd.com>
2026-05-11 16:11:49 -07:00
Wentao YeNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
d7af6b34d8 [Model Runner V2] Bug fix: logprob dtype int64/int32 issue (#41761)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-11 21:55:43 +00:00