iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
16,720 Commits 3 Branches 0 Tags
f5d3dc7115cf77472ba5e274f6becbbeddbf4bd5
Commit Graph
16720 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael GoinandClaude Opus 4.7 f5d3dc7115 [Model Runner v2] Support update_config (#42783) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 10:26:07 -04:00
vllm-agentLuka GovedičZhewen Li
 		1ac10f159a Revert "[torch.compile] Add patch for fullgraph compilation" (#42686) (#42913) 
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
 2026-05-18 09:02:51 -04:00
liranschourOr Ozerimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e5417657e5 [KV Connector][Offloading] Flush all pending jobs on last step (#42611) 
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: liranschour <liranschour@users.noreply.github.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-18 12:59:42 +00:00
xiangdong 2e40faf08b [XPU][CI] Temporarily skip test_moe_lora_align_block_size_mixed_base_and_lora[1] in Intel GPU CI (#42954) 
Signed-off-by: zengxian <xiangdong.zeng@intel.com>
 2026-05-18 20:34:48 +08:00
Nicolò Lucchesi 69c91d010a [MRv2] Default to MRv1 when a connector is present (#42955) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2026-05-18 20:34:16 +08:00
roikoren755 737bfa3a43 [Bugfix][Hybrid][NemotronH] Fix mamba_cache_mode=all + speculative decoding crash (#41233) 
Signed-off-by: Roi Koren <roik@nvidia.com>
 2026-05-18 14:54:00 +03:00
Kfir Toledo e414e1f1c0 [Bugfix][KV Offload] count appended GPU blocks in store group_sizes (#42945) 
Signed-off-by: Kfir Toledo <kfir.toledo@ibm.com>
 2026-05-18 11:36:02 +00:00
inisisandYongye Zhu df852ed503 fix: remove unused norm for dpskv4 (#41710) 
Signed-off-by: inisis <desmond.yao@buaa.edu.cn>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-05-18 18:33:29 +08:00
Yuwen Zhou 88a860d754 [CPU] Add MXFP4 W4A16 MoE support (#41922) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com>
 2026-05-18 03:04:45 -07:00
Tianmu Liandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> cac81b6eda [CPU Backend] Improve cpu thread utilization (#42666) 
Signed-off-by: Li, Tianmu <tianmu.li@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-18 03:04:41 -07:00
Li, Jiang b4601ad43f [CPU] Add fused GDN support for AMX CPU platform (#42707) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2026-05-18 03:04:36 -07:00
Jee Jee Li 2267f70070 [Kernel] Pack topk id/weights triton kernel (#42527) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-05-18 03:04:31 -07:00
Tony LinandLi, Jiang <jiang1.li@intel.com> 965d076148 [CPU] Specify required KV cache layout for CPU attention backend (#42740) 
Signed-off-by: Tony Lin <tony.lin@intel.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-05-18 17:38:54 +08:00
wenjun liuKunshang Jimergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		c38bed4248 delete xpu ci (#42582) 
Signed-off-by: wenjun.liu <wenjun.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-05-18 16:36:45 +08:00
Xin Yang 998714b21b [Perf] Add do_not_specialize in fused FP8 RoPE kernel (#42849) 
Signed-off-by: Xin Yang <xyangx@amazon.com>
 2026-05-18 01:32:46 -07:00
Harry Mellor 9537542537 Revert checkpoint specific workaround in Transformers modelling backend (#42923) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-18 17:31:06 +09:00
Rishapveer Singh 5ab6d1b3fd [Model] [Perf] Use flatten for Qwen3.5's GDN output projection (#42311) 
Signed-off-by: Rishapveer Singh <singhrishapveer@gmail.com>
 2026-05-18 16:14:36 +08:00
Jee Jee Ligemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Roger Wang
 		7d5b033782 [LoRA] Support 2D and 3D MoE LoRA adapter at the same time (#42242) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-18 15:22:26 +08:00
Nguyễn Thế DuyJee Jee LiJee Jee Li
 		e3aeee5ff8 [Bugfix] moe lora align kernel grid (#40131) 
Signed-off-by: TheDuyIT <nduy250299@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: dtnguyen <dtnguyen@nvidia.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-05-18 00:17:53 -07:00
Harry Mellor c1f7854342 Improve logging when docs build is skipped (#42929) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-05-18 06:33:32 +00:00
gaozihao-shy 23c15acd77 [BugFix] Kimi-K2.5: skip vision tower dtype conversion when using quantization (#42869) 
Signed-off-by: gaozihao-shy <gaozihao-shy@users.noreply.github.com>
Signed-off-by: gaozihao <gaozihao3@huawei.com>
 2026-05-18 05:07:16 +00:00
Andreas Karatzas b50646e5ef [ROCm][CI] Stabilize ROCm pooling and multimodal CI (#42909) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-18 03:57:59 +00:00
Soyaazz 990f49bdcb [MM][CG] Enable encoder Cudagraph for Step3VL (#42224) 
Signed-off-by: JisoLya <523420504@qq.com>
Signed-off-by: Soyaazz <523420504@qq.com>
 2026-05-17 20:19:13 -07:00
AlecandOpenAI Codex 107210442d [CI] Add NIXL EP import canary (#42567) 
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-05-17 19:11:46 -07:00
Yiliu Dongandqianlihuang 03ddc1c9bc [Perf] Wire silu_and_mul_per_block_quant into TritonFP8MoE (MiniMax-M2) (#42497) 
Signed-off-by: qianlihuang <yiliu.dong@qq.com>
Signed-off-by: Yiliu Dong <91178480+qianlihuang@users.noreply.github.com>
Co-authored-by: qianlihuang <yiliu.dong@qq.com>
 2026-05-17 21:57:04 -04:00
Luka Govedič 966903eb93 [torch.compile] Add patch for fullgraph compilation (#42686) 
Signed-off-by: Luka Govedič <luka.govedic@gmail.com>
 2026-05-17 19:49:16 +00:00
TJian 599e75f432 [ROCm] [Bugfix] Fix DeepSeek V4 Functionality and Accuracy (#42810) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-05-17 12:18:50 -04:00
Taneem Ibrahim 1c8e9c0399 Refactor: Pass num_labels explicitly to PoolerClassify instead of reading from global config (#42851) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-05-17 14:40:21 +00:00
zofiaandKunshang Ji 0fa888465e [XPU] fix weight scale shape (#42725) 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-17 16:55:10 +08:00
liuzhenwei ff712f6447 [MRV2][XPU] add Model Runner V2 log (#42710) 
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
 2026-05-17 04:15:50 +00:00
Qi Zhou 504a26ce2b Support bf16 for mamba ssm cache (#41680) 
Signed-off-by: Qi Zhou <qizzzh@google.com>
 2026-05-16 17:54:58 -07:00
weizhoublue a94189295b Fix Weight loading for Qwen3.5-MTP and Qwen3-VL using runai_streamer (#42716) 
Signed-off-by: weizhoublue <weizhou.lan@daocloud.io>
 2026-05-16 17:54:27 -07:00
Artem PerevedentsevandVadim Gimpelson 0867497368 [CI/Build] Bump flashinfer to v0.6.11.post2 (#41711) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
 2026-05-16 14:55:12 -07:00
Zhewen LiZhewen LiClaude Opus 4.7
 		36e74c9ea4 [KV Connector] Support disk offloading in MooncakeStoreConnector (#42689) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 13:34:15 -07:00
Taneem Ibrahim 787bc0d031 Add unit tests for pooler activation functions (#42824) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-05-16 14:58:16 -04:00
weizhoublue d1586e1a12 Fix: Propagate pinned model revisions into Ultravox secondary weight loading (#42830) 2026-05-16 17:02:54 +00:00
Jiangyun Zhu 8a56da3845 [Experimental] Breakable CUDA graph (#42304) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-05-16 22:04:12 +08:00
Andreas Karatzas 4db300e95f [ROCm][CI] Removed problematic command override mechanism (#42807) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-05-16 17:35:05 +08:00
Zhewen LiZhewen Likhluu
 		657b42b592 [Docker][KVConnector] Build mooncake-transfer-engine from source (#42114) 
Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: khluu <khluu000@gmail.com>
 2026-05-16 00:26:25 -07:00
Jee Jee Li 32b7177909 [LoRA][Bugfix] Dedup LoRA wrapping for modules referenced from multiple attribute paths (MoE gate) (#42757) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-05-16 11:22:35 +08:00
DustHunterQwen-Coderwang.yuqi
 		39c67d714e fix: add API key authorization to /v2 endpoints (#42594) 
Signed-off-by: DustHunter <dusthunter@126.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-05-16 01:29:27 +00:00
Viktor Pusandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> 87a2adcb43 [Misc] Add common random prefix option to structured-output serving benchmark (#41632) 
Signed-off-by: Viktor Pus <viktorpus@tenstorrent.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-16 00:44:48 +00:00
Michael Goin 852f567444 [Bugfix] Respect explicit --kv-cache-dtype over checkpoint kv_cache_scheme (#42782) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-15 17:15:52 -07:00
Michael Goin b2a27b82d9 [Kernel][UX] Add --linear-backend arg for linear kernel selection (#39538) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-15 17:07:39 -07:00
Keyi Li d0921bafef [Bugfix] Unwrap VLM wrappers for EPLB on Model Runner V2 (#42706) 2026-05-16 07:20:33 +08:00
rasdaniOpenAI CodexRoger Wang
 		1ccdf87507 [Bugfix] Fix layerwise reload alias-buffer corruption (#42481) 
Signed-off-by: rasdani <73563550+rasdani@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-05-15 15:20:53 -07:00
Rita Brugarolas bd9dbe6060 [ROCm][Bugfix] Fix fused_mla_dual_rms_norm for AITER API rename _fused_qk_rmsnorm (#42606) 
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
 2026-05-15 14:50:03 -06:00
Michael GoinandClaude de2d76f352 [Build] Switch CUDA 12.9 wheel builds to PyTorch manylinux_2_28 base (#41668) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-15 13:46:16 -07:00
Sergei SkvortsovandMichael Goin 9a7a273dfe Add HumanEval and GSM8K benchmarks to datasets (#42648) 
Signed-off-by: southfreebird <yvorott@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-05-15 13:01:21 -07:00
Lanze LiuandMatthew Bonanni b2c58ee942 [FlashAttn] Fix supports_kv_cache_dtype() accepting unhandled fp8 kv-cache dtype variants (#42685) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-15 15:34:59 -04:00