iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,489 Commits 3 Branches 0 Tags
9efdddca283cc0eb7c37fa49c4a9d1c9bf59ec4e
Commit Graph
5452 Commits
Author SHA1 Message Date
shaharmor98andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f9f770ca0b fix nixl side-channel host selection (#41806) 
Signed-off-by: Shahar Mor <smor@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-11 07:40:37 +00:00
Jee Jee Li 05d610e5cd [CI/Build] Reduce LoRA model tests. (#42266) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-11 14:49:08 +08:00
wangxiyuan 5536fc0c01 [Misc] Replace mamba_type string literals with MambaAttentionBackendEnum (#41188) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-05-11 03:59:36 +00:00
gnovackandgeorge 171019ab19 add fused mhc_post_pre kernel (#41536) 
Signed-off-by: george <george@inferact.ai>
Co-authored-by: george <george@inferact.ai>
 2026-05-10 19:56:52 -07:00
Haoqing Wang 879a8c3180 Fix Molmo2 image token metadata (#42162) 
Signed-off-by: Haoqi Wang <78337154+hqhq1025@users.noreply.github.com>
 2026-05-11 01:19:21 +00:00
bnellnmJackmin801ClaudeRobert ShawRobert ShawJackmin801
 		1b57eb41f2 [MoE] Move various experts classes to fused_moe/experts/ (#41979) 
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com>
 2026-05-11 07:54:33 +08:00
Ronen Schaffer e175192d33 [KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() (#41366) 
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
 2026-05-10 15:09:25 +03:00
Ethan Feng a2c9d548d7 [Docs] Fix broken local links (#42160) 
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com>
 2026-05-10 01:15:38 -07:00
Mohammad Miadh Angkad efd0e7789d Fix mypy failure on main (#42197) 
Signed-off-by: Mohammad Miadh Angkad <MAngkad.BSDSBA2027@aim.edu>
 2026-05-10 07:55:57 +00:00
abdulrahman-cohereCursor AgentNicolò Lucchesi
 		a5d0a5afba [Frontend][Bugfix] Abort ASR engine requests on cancellation (#41266) 
Signed-off-by: abdulrahman-cohere <abdulrahman.abdulrazzag@cohere.com>
Signed-off-by: <>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
 2026-05-09 23:51:11 -07:00
Andreas Karatzas f2840120f6 [ROCm][CI] Fix NIXL spec-decode acceptance startup and diagnostics (#41313) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-10 14:50:16 +08:00
Dao007forever 3f5bd482f5 [Bugfix][KV Transfer][NIXL] Notify P node on pre-admission rejection to free stranded KV blocks (#41269) 2026-05-09 22:52:09 -07:00
Andreas Karatzas fb1ac806c5 [ROCm][CI] Stabilize ROCm shutdown and distributed compile CI (#41573) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-10 03:47:40 +00:00
Christian VanandChristian Van 0d382ecde8 Handle optional bool-or-string CLI args in get_kwargs (#40951) 
Signed-off-by: Christian Van <cvan20191@gmail.com>
Co-authored-by: Christian Van <cvan20191@gmail.com>
 2026-05-09 19:47:21 -07:00
Isotr0py 1029e5ef28 [CI/Build] Use modelscope's international site for regression test (#42176) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-05-09 19:47:09 -07:00
Wang XingranandHongjian Zhang 0b272a6e01 [Bugfix] Fix SP pass for multimodal models and PP+SP residual handling (#33322) 
Signed-off-by: Xingran Wang <wangxingran123456@outlook.com>
Signed-off-by: Hongjian Zhang <hirokenovo@gmail.com>
Co-authored-by: Hongjian Zhang <hirokenovo@gmail.com>
 2026-05-09 19:44:16 -07:00
baonudesifeizhaiandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bc5fdc1e6a Add NVFP4 all-gather GEMM fusion for AsyncTP (#41882) 
Signed-off-by: roG0d <baonudesifeizhai@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-10 01:13:22 +00:00
Wentao Ye ea0e501bb1 [KV Connector] Remove compat support for pre-v0.12.0 constructor signatures without KVCacheConfig (#39832) 
The v0.12.0 release contained initial support for HMA in KV Connectors. As part
of these changes, a KVCacheConfig argument was added to KV connector
constructors. Backwards compatibility support for out-of-tree connectors was
included in this change, with a very prominent warning. See #25712 and #27887.

Since the warning has been around for over 5 months, we can safely remove
the support of it.

Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-09 23:39:46 +00:00
Juhi Mittal 7a2b596982 [Quantization] Add ModelOpt NVFP4 W4A16 (4-bit weights, fp16/bf16 activations) support (#41769) 
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
 2026-05-09 21:15:50 +00:00
SoluMilken 25abddc1a5 [BugFix] Fix Gemma4 'layers.0.moe.experts.0.down_proj_packed' KeyError issue (#40708) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-05-09 17:20:44 +00:00
Nicolò LucchesiandZhanqiuHu 171d59ae8d [Bugfix][PD] Fix DSv4 Disaggregated (#41957) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: ZhanqiuHu <zhu@redhat.com>
 2026-05-09 16:48:24 +00:00
Thien Tran 530d371302 [DSv4] Improved fused Indexer Q quant kernel (#41428) 2026-05-09 01:20:32 -07:00
Jee Jee Li ecd0b60aad [LoRA] Initial EP support for LoRA (#40867) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-09 00:31:23 -07:00
Rishapveer Singh f6490a2841 [Bugfix] Preserve leading/trailing whitespace in GLM non-streaming tool parser (#42026) 
Signed-off-by: Rishapveer Singh <singhrishapveer@gmail.com>
 2026-05-08 21:49:15 -07:00
Terrence ZhaoandCursor a2812becd6 [Models] Cohere Eagle + fix to Cohere MoE (#42078) 
Signed-off-by: Terrencezzj <terrence@cohere.ai>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-08 21:46:26 -07:00
David Zhenggemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Andreas KaratzasKunshang Ji
 		df2636a9d8 [Bugfix] Fix LOGITPROC_SOURCE_ENTRYPOINT test to use spawn-compatible dist-info registration for XPU/ROCm (#42040) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
Signed-off-by: David Zheng <153074367+dzhengAP@users.noreply.github.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-09 12:32:04 +08:00
haosdent e934e459e6 [CI][Bugfix] Make test_gpt2_cache_hit observable across V1 EngineCore (#42037) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-09 11:53:15 +08:00
David Zheng 845ca327ce [Bugfix] Fix test_whisper distributed test process handling (#42038) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
 2026-05-09 11:37:21 +08:00
Lucas Wilkinson b1728c1e66 [Attention][Cleanup] Remove tree attention (#42121) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2026-05-08 18:36:19 -07:00
Sumanth R Hegde e3b65a5ba0 [feat] Add explicit /start_weight_update and /finish_weight_update APIs for weight transfer (#39212) 2026-05-08 18:03:33 -07:00
Roy Wang 60851b1d22 [Bugfix][KV Transfer] Reject NixlConnector + expandable_segments:True (#41237) 2026-05-08 16:47:33 -07:00
Michael Goin 8bcd8a260c [Bugfix] Fix FlashInfer CUTLASS MXFP4-MXFP8 MoE by restoring swizzled scale (#42089) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-08 15:59:06 -07:00
John Calderon 8a2fc80b84 [CUDA][CUTLASS] Enable cutlass scaled mm for non-compatible sizes (#41868) 
Signed-off-by: John Calderon <jcalderon@nvidia.com>
 2026-05-08 15:58:05 -07:00
David OyandClaude dbd86a67e3 [Bugfix][Gemma4] Fix infinite loop and array boundary issues in tool parser (#41991) 
Signed-off-by: David Oy <david.oy@baseten.co>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-08 17:24:37 -04:00
Hiroaki Mikami 90f145aaf7 [Models][Gemma3/Gemma4] Support hidden_act variants in gated MLP (#40588) 
Signed-off-by: Hiroaki Mikami <hiroaki8270+github@gmail.com>
 2026-05-08 11:29:11 -07:00
haosdent 160858cba4 [CI][Bugfix] Surface subprocess output in spawn_new_process_for_each_test (#41943) 
Signed-off-by: haosdent <haosdent@gmail.com>
 2026-05-08 10:39:37 +02:00
David Zheng 1acd67a795 [Bugfix] Fix XPU/ROCm compatibility in spawn_new_process_for_each_test (#41895) 
Signed-off-by: dqzhengAP <dqzheng1996@gmail.com>
 2026-05-08 00:47:22 -04:00
Yanan CaoandClaude Sonnet 4 0b99971352 [Kernel][Helion] Optimize Helion config parsing latency (#40850) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com>
 2026-05-07 20:27:34 -07:00
Shrinav LokaClaude Opus 4.6wang.yuqimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		cd58e30872 [Perf] Use numpy zero-copy path for embedding float response serialization (#41681) 
Signed-off-by: Shrinav Loka <lokashrinav@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-07 19:42:21 -07:00
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 1d694e78c9 [Examples][last/6] Resettle examples. (#41084) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-07 19:42:12 -07:00
haosdentandRoger Wang 5f6a02812a [CI][Bugfix] Fix failure CI step "PyTorch Fullgraph Smoke Test" (#41953) 
Signed-off-by: haosdent <haosdent@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-07 19:41:56 -07:00
JonathanandChauncey 50f2db2555 add: LFM2/2.5 Tool Parser (#39243) 
Signed-off-by: Jonathan Buchanan <jonathan.buchanan@liquid.ai>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-05-08 09:58:17 +08:00
Kyle Sayers c1819ca283 [Compressed Tensors] Allow configs with non-explicit ignores (#41965) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-05-07 14:03:45 -07:00
TomerBN-Nvidia 8189a15914 [Core] Replace routing replay with device cache and async D2H pipeline (#39917) 
Signed-off-by: Tomer Barnatan <tbarnatan@nvidia.com>
 2026-05-07 11:24:56 -07:00
Flora Feng 8eb401134e [Refactor] Consolidate required/named tool_choice streaming into DelegatingParser (#41876) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-07 09:50:59 -07:00
Chaojun Zhang 805e9f7b77 [XPU] Fix lora bugs & enable UTs under tests/lora (#38206) 
Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com>
 2026-05-07 05:58:00 -07:00
SeongJun Leeandlesj0610 f650ace6de [MM][Gemma4] Use video profiling hints in encoder budget (#41837) 
Signed-off-by: lesj0610 <lesj0610@users.noreply.github.com>
Co-authored-by: lesj0610 <lesj0610@users.noreply.github.com>
 2026-05-07 05:46:04 -07:00
Li, Jiang b3945cc316 [CPU] Bump up to the latest CPU kernels (#41924) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-05-07 05:45:59 -07:00
Yuwen Zhou 713b28bd0b [CPU] Add FP8 W8A16 MoE support (#41314) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
 2026-05-06 23:17:07 -07:00
Hongxia YangandTJian 20cac26b19 [ROCm] Enable SimpleCPUOffloadConnector on ROCm (#40549) 
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2026-05-06 20:52:02 -07:00