iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
16,564 Commits 3 Branches 0 Tags
71bcd02ef369866403c8a8bc03f7dc86e4f68f46
Commit Graph
5488 Commits
Author SHA1 Message Date
Nicolò Lucchesi 71bcd02ef3 [Bugfix][PD] Fix multi-node TP (TP>8) (#39907) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-12 22:20:57 -07:00
bnellnm 18f6bf5a21 [MoE Refactor] Add sequence parallel tests to test_moe_layer.py (#41299) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-05-12 21:52:19 -04:00
+3
Chao Leileichao.lcivaniumaoshen524Dao007foreverTeng MaPz1116foraxeSkywalker-EPfems14jianzsbaxingpiaochongmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ebeb09d822 [KV Transfer] Add MooncakeStoreConnector for KV cache offloading via Mooncake distributed store (#40900) 
Signed-off-by: leichao.lc <leichao.lc@antgroup.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: leichao.lc <leichao.lc@antgroup.com>
Co-authored-by: ivanium <yifanqiao@inferact.ai>
Co-authored-by: aoshen524 <aoshen@inferact.ai>
Co-authored-by: Dao007forever <daole@inferact.ai>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
Co-authored-by: Pz1116 <zpbzpb123123@gmail.com>
Co-authored-by: foraxe <1055696449@qq.com>
Co-authored-by: Skywalker-EP <173423846@qq.com>
Co-authored-by: fems14 <1804143737@qq.com>
Co-authored-by: jianzs <zheng.shoujian@outlook.com>
Co-authored-by: baxingpiaochong <771405853@qq.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-12 16:09:10 -07:00
Nick Hill fe8b42e80c [CI] Fix test_async_scheduling.py flakiness (#42455) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-05-12 21:38:32 +00:00
Giancarlo Delfin fe5b4e0fe7 [Model Runner V2] Apply synthetic mode to probabilistic rejection sampler (#41035) 2026-05-12 13:37:03 -07:00
KaivalyaMDabhadkar 67c89fe40a [Model][Bugfix] Fix Step3-VL image_embeds input path (#42333) 
Signed-off-by: Kaivalya Dabhadkar <kdabhadkar@nvidia.com>
 2026-05-12 18:47:55 +00:00
bnellnmandRobert Shaw d9b4990783 [MoE Refactor] EPLB refactoring for FusedMoE (#41055) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-05-12 14:16:31 -04:00
bnellnmRobert ShawRobert Shaw
 		4d591db470 [MoE Refactor] Introduce RoutedExperts alias for FusedMoE and don't store SharedExperts in MK (#40735) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-05-12 13:37:44 -04:00
yzong-rh 6ff7405b81 [Bugfix] [Frontend] Responses API, fix merging of messages (#42189) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Yifan <yzong@redhat.com>
 2026-05-12 16:09:59 +00:00
Yan Ru Pei bcb9c133ba feat(kv-events): emit KV cache metadata (#40984) 
Signed-off-by: PeaBrane <yanrpei@gmail.com>
 2026-05-12 15:58:48 +00:00
Martin Hickey 418ba8ef14 [kv_offload][BugFix] Fix store deferral (#41945) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-05-12 18:04:44 +03:00
Avishek GoswamiandAvishek Goswami 289cee0473 [vLLM IR] Minor improvements (#39362) (#39558) 
Signed-off-by: Avishek Goswami <avishek.goswami@ibm.com>
Co-authored-by: Avishek Goswami <avishek.goswami@ibm.com>
 2026-05-12 10:58:36 -04:00
shanjiaz 6ccb10d794 Added peagle speculators support (#41826) 
Signed-off-by: shanjiaz <zsjwpianpian@gmail.com>
 2026-05-12 07:55:57 -07:00
tc-mbandcaitianchi 7a9cc5e7f0 [Model] Support MiniCPM-V 4.6 (#41254) 
Signed-off-by: caitianchi <caitianchi@tc-mb.com>
Signed-off-by: tc-mb <157115220+tc-mb@users.noreply.github.com>
Co-authored-by: caitianchi <caitianchi@tc-mb.com>
 2026-05-12 14:28:10 +00:00
zhanqiuhuandNickLucche 593d5a4033 [Bugfix] Fix mismatched kernel-per-logical blocks in NIXL HMA transfer (#42097) 
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2026-05-12 15:53:30 +02:00
bnellnm 6427603ae8 [MoE Refactor] Move remaining experts classes to experts directory (#42334) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2026-05-12 09:19:46 -04:00
bnellnmRobert ShawRobert Shaw
 		206eaed08d [MoE Refactor] Move expert map related code into ExpertMapManager class (#41046) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
 2026-05-12 09:18:27 -04:00
Dipika Sikka a7b801e26d [MXFP4] Support for linear layers + compressed-tensors integration (#41664) 2026-05-12 07:49:33 -04:00
haosdent fc8bf6eedb [CI] De-flake Language Models Test (Extended Generation) test_models(False-False-5-32-bigcode/starcoder2-3b) (#42392) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-12 10:46:48 +00:00
liuzhenwei 07a40ede19 [UT][XPU] fix test_parallel_sampling due to global random state (#42388) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-05-12 18:03:23 +08:00
Kevin H. Luu 1ff9d33535 [CI] Migrate remaining B200 jobs to b200-k8s with test fixes (#42387) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-05-12 02:00:37 -07:00
chazenandChauncey 7f65f84428 [Bugfix] Fix empty channel/recipient in harmony for /v1/responses (#35540) 
Signed-off-by: kg6-sleipnir <christopherhazen42@gmail.com>
Signed-off-by: chazen <45186108+kg6-sleipnir@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-12 08:45:51 +00:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> d37e25ffbe [Frontend] Consolidate Speech to Text entrypoints. (#42370) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-12 07:06:57 +00:00
ZiTian Zhao 630492da30 [Fix] Gemma4 Mixed-Resolution Image Co-Batching Crash (#42217) 
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
 2026-05-12 03:13:03 +00:00
Wentao YeNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d7af6b34d8 [Model Runner V2] Bug fix: logprob dtype int64/int32 issue (#41761) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-11 21:55:43 +00:00
wang.yuqi a0dc7a0f36 [CI] Consolidate Speech to Text tests (#42274) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-11 19:50:17 +00:00
frida-anderssonMarkus HartikainenCursorChuan LiLuka GovedičFrida AnderssonTJian
 		a721315488 [ROCm][Perf] Fix RMSNorm+Quant fusion for gfx950 (non-fnuz) (#41825) 
Signed-off-by: Frida Andersson <fanderss@amd.com>
Signed-off-by: Chuan Li <chuali@amd.com>
Co-authored-by: Markus Hartikainen <markus.hartikainen@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Chuan Li <chuali@amd.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Frida Andersson <frida-andersson@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-11 15:00:51 -04:00
Hexiang Wang 7863fff6e5 [ROCm][DSv4] implement flash sparse mla with triton kernels (#41812) 
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
 2026-05-11 09:27:11 -07:00
Wentao Ye 0d453e2336 [Perf] Batch invariance with Cutlass fp8 support, 28.9% E2E latency improvement (#40408) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-05-11 12:20:58 -04:00
Martin Hickeyandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 4955990f1b [kv_offload] Move FilterReusedOffloadingManager logic to CPUOffloadingManager (#41727) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-11 18:09:29 +03:00
Thien TranandYongye Zhu 724ed2fc35 [DSv4] Improved dequant gather K cache kernel (#42236) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-11 10:41:12 -04:00
Rohan PotdarandElizaWszola a51376b3f0 [Performance][DSR1]: Fused RoPE+KVCache+q_concat for MLA (#40392) 
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>
 2026-05-11 14:10:50 +00:00
Noa Neria ac062147fa Avoid silent weights corruption when loading Nemotron Nano VL with reusable-buffer loaders like runai distributed streaming (#42244) 
Signed-off-by: Noa Neria <nneria@nvidia.com>
 2026-05-11 12:03:14 +00:00
Chauncey 617239b70c [Frontend]Responses API supports chat_template_kwargs (#42272) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-11 11:59:39 +00:00
Nicolò Lucchesi 5672d100ed [KV Connector][NIXL][Bugfix] Fix NIXL handshake failures not honoring kv_load_failure_policy (#40364) 
When NIXL handshake fails (e.g., due to compatibility hash mismatch
between prefill and decode instances), requests fail with "engine dead"
error instead of gracefully falling back to local recomputation as configured
by kv_load_failure_policy='recompute'.

Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:37:21 +00:00
Nicolò Lucchesi 770e9bd6b3 [Nixl][PD] Lease renewal TTL KV blocks on P (#41383) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-11 09:27:30 +00:00
shaharmor98andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f9f770ca0b fix nixl side-channel host selection (#41806) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-11 07:40:37 +00:00
Jee Jee Li 05d610e5cd [CI/Build] Reduce LoRA model tests. (#42266) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-11 14:49:08 +08:00
wangxiyuan 5536fc0c01 [Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-05-11 03:59:36 +00:00
gnovackandgeorge 171019ab19 add fused mhc_post_pre kernel (#41536) 
Signed-off-by: george <george@inferact.ai>
Co-authored-by: george <george@inferact.ai>
 2026-05-10 19:56:52 -07:00
Haoqing Wang 879a8c3180 Fix Molmo2 image token metadata (#42162) 
Signed-off-by: Haoqi Wang <78337154+hqhq1025@users.noreply.github.com>
 2026-05-11 01:19:21 +00:00
bnellnmJackmin801ClaudeRobert ShawRobert ShawJackmin801
 		1b57eb41f2 [MoE] Move various experts classes to fused_moe/experts/ (#41979) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
 2026-05-11 07:54:33 +08:00
Ronen Schaffer e175192d33 [KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() (#41366) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-05-10 15:09:25 +03:00
Ethan Feng a2c9d548d7 [Docs] Fix broken local links (#42160) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-10 01:15:38 -07:00
Mohammad Miadh Angkad efd0e7789d Fix mypy failure on main (#42197) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
 2026-05-10 07:55:57 +00:00
abdulrahman-cohereCursor AgentNicolò Lucchesi
 		a5d0a5afba [Frontend][Bugfix] Abort ASR engine requests on cancellation (#41266) 
Signed-off-by: abdulrahman-cohere <abdulrahman.abdulrazzag@cohere.com>
Signed-off-by: <>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-09 23:51:11 -07:00
Andreas Karatzas f2840120f6 [ROCm][CI] Fix NIXL spec-decode acceptance startup and diagnostics (#41313) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-10 14:50:16 +08:00
Dao007forever 3f5bd482f5 [Bugfix][KV Transfer][NIXL] Notify P node on pre-admission rejection to free stranded KV blocks (#41269) 2026-05-09 22:52:09 -07:00
Andreas Karatzas fb1ac806c5 [ROCm][CI] Stabilize ROCm shutdown and distributed compile CI (#41573) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-10 03:47:40 +00:00
Christian VanandChristian Van 0d382ecde8 Handle optional bool-or-string CLI args in get_kwargs (#40951) 
Signed-off-by: Christian Van <cvan20191@gmail.com>
Co-authored-by: Christian Van <cvan20191@gmail.com>
 2026-05-09 19:47:21 -07:00