iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
10,523 Commits 3 Branches 0 Tags
a5464dcf92bba8dfd052fc79bfc40e08aee515d9
Commit Graph
7272 Commits
Author SHA1 Message Date
Kyle SayersandMichael Goin a5464dcf92 [Compressed Tensors] Always clone output for compile robustness (#26849) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-16 19:29:59 +00:00
Mandy Li ac3ed5a815 Support block size of 256 used by Intel HPU (#26883) 
Signed-off-by: mandy-li <mandy.j.li@intel.com>
 2025-10-16 15:10:57 -04:00
Andrew Xia e6ba2000ae [gpt-oss][1/N] EZ: refactor serving_responses for modularity (#26948) 
Signed-off-by: Andrew Xia <axia@meta.com>
 2025-10-16 18:44:06 +00:00
Harry Mellor aa255ff55a Support set in the CLI generation (#27031) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-16 18:07:18 +00:00
ZiTian Zhao 7bb736d00e Fix Qwen2.5 VL image grid docstring (#27033) 
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
 2025-10-16 09:57:36 -07:00
Jee Jee Li 9f4e30904b [Model] Fix Qwen3VL mm mapping (#27027) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-16 09:45:59 -07:00
rongfu.leng 5afd3276df [Feature] Add process_weights_after_loading to AttentionImpl (#26870) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-10-16 08:02:30 -07:00
Mark McLoughlin 4a510ab487 [NIXL] Improve request_finished() debug logs (#25665) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-10-16 15:55:17 +02:00
Cyrus Leung 334535b6fb [Benchmark] Show E2EL by default for pooling models (#27014) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 12:47:09 +00:00
bogdanmandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> dcbb3f1871 [Bugfix] Correct LayerNorm epsilon parameter in modernbert.py (#27008) 
Signed-off-by: bogdanm <152898065+bogdan01m@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-10-16 12:27:44 +00:00
Sungjae LeeandMengqing Cao 00417f4e44 [MISC] fix import violations for re and triton modules (#26654) 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
 2025-10-16 03:38:27 -07:00
CSWYF3634076 e51928793e [Model][Bugfix] fix ernie45 vl run failed from shared experts optimization (#26885) 
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
 2025-10-16 03:37:35 -07:00
Cyrus Leung d2740fafbf [Chore] Separate out vllm.utils.collections (#26990) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 08:35:35 +00:00
Cyrus Leung 17838e50ef [Benchmark] Use truncation by default for pooling benchmarks (#26992) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 16:02:39 +08:00
Akash kaothalkarandAkash Kaothalkar f7d318de2b [Hardware][CPU][PowerPC]Disable torch.compile() in toptopk sampling (#26987) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
 2025-10-15 22:36:59 -07:00
Bram WastiandWentao Ye 7d8975de84 Deepseek-v3 Batch Invariant on 8xH100 (#26609) 
Signed-off-by: Bram Wasti <bwasti@meta.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2025-10-15 22:06:02 -07:00
Vadim Gimpelson 785d8b6410 [PERF] Qwen3-next MTP speedup (change bool mask indexing to index_select / index_copy to reduce d2h) (#26437) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
 2025-10-16 12:18:31 +08:00
Cyrus Leung f6cdc9a02f [Chore] Rename utils submodules (#26920) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-16 03:58:13 +00:00
Richard Zou 9b6504c307 [BugFix] Work around graph partition x torch.compile cache issue (#26956) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-10-15 20:06:11 -07:00
Angela Yi e19b16dde6 [bugfix] Fix SP + PP without specifying compile size (#26955) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-10-15 20:05:33 -07:00
ahao-anyscale 582f2c6be7 [BUG] Allow runai_streamer_sharded in config check (#26958) 
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
 2025-10-15 20:05:14 -07:00
kliuaevalarLipTJian
 		1317034379 [ROCm][FEAT] Fuse DeepSeek shared experts into AITER fused_moe ops (#24097) 
Signed-off-by: chenjun <junchen2@amd.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: valarLip <103567126+valarLip@users.noreply.github.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
 2025-10-16 10:41:34 +08:00
InChang JeongandUSER 0ecc553ee6 [Bugfix] reasoning_parser parameter handling in run_batch.py (#26225) 
Signed-off-by: inc-jeong <inc.jeong@navercorp.com>
Signed-off-by: InChang Jeong <inc.jeong@navercorp.com>
Co-authored-by: USER <user@AL02367916.local>
 2025-10-16 10:24:05 +08:00
felixzhu555 f96bc3649c [Qwen3-Next] Add tuned MoE config for Qwen3-Next FP8 on H100 tp2 (#26887) 
Signed-off-by: Felix Zhu <felixzhu555@gmail.com>
 2025-10-15 18:55:05 -07:00
Adrian AbeytaandLuka Govedič 0a9ef0cfce Move query quantization to attention layer for Flashinfer & Triton. (#26534) 
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Adrian Abeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2025-10-15 19:01:38 -04:00
Wentao Ye e5b438a247 [Bug] Temporally Disable VLLM_ALLREDUCE_USE_SYMM_MEM by Default (#26925) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-15 16:18:50 -04:00
XiaobingZhangandMichael Goin 0b99f5d302 support flashinfer_fp4 moe for 5090 gpu (#26669) 
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-10-15 15:06:47 -04:00
Kaixi Houandmgoin de92d916fe [NVIDIA] Add support for cudnn fp4 gemm via flashinfer (#26107) 
Signed-off-by: kaixih <kaixih@nvidia.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-10-15 13:53:00 -04:00
XiaobingZhang d796375258 [ModelOpt] Remove NVFP4 MoE K%16==0 constraint (#26891) 
Signed-off-by: XiaobingSuper <xiaobingzhangupc@gmail.com>
 2025-10-15 13:06:17 -04:00
Sam/SamuelandHarry Mellor 14f8456344 [Feature]: Use pydantic validation in observability.py config (#26637) 
Signed-off-by: Samuel Wu <cernunnos1710@gmail.com>
Signed-off-by: Sam/Samuel <57896620+cern1710@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-15 16:44:03 +00:00
Pradeep Dasigi 4794c2bd92 Olmo 3 tool parser and tests (#26143) 
Signed-off-by: Pradeep Dasigi <pradeepd@allenai.org>
 2025-10-15 16:36:12 +00:00
Harry Mellor d3cbaa08dc Lower sevarity of log when model info cache misses due to exception (#26917) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-15 09:01:09 -07:00
Cyrus Leung 828523ad8e [Chore] Separate out vllm.utils.async_utils (#26913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 15:33:00 +00:00
Cyrus Leung 136a17fe6e [Chore] Separate out vllm.utils.func (#26904) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 13:03:58 +00:00
Boyuan FengandHarry Mellor f57438338d [BugFix] Patch inductor memory plan logic (#26878) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-15 12:51:45 +00:00
wangxiyuan 8f4b313c37 [Misc] rename torch_dtype to dtype (#26695) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-15 12:11:48 +00:00
Cyrus Leung f93e348010 [Misc] Remove isort and yapf ignores (#26888) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-15 12:09:03 +00:00
wang.yuqi f54f85129e [Model][2/N] Improve all pooling task | Support multi-vector retrieval (#25370) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-15 11:14:41 +00:00
li2haipengandJee Jee Li d4d1a6024f [Lora]Load tuned multi-lora kernel configs from json files (#26319) 
Signed-off-by: li2haipeng <44383182+li2haipeng@users.noreply.github.com>
Signed-off-by: Haipeng Li <li2haipeng@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-10-15 09:45:14 +00:00
wangxiyuan db1764e4e0 [Platform] allow platform to init dp group (#22243) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-10-15 02:32:17 -07:00
Jialin Ouyang 7f83b4ee8e [Easy] Get rid of unnecessary paraenthesis in kv_cache_manager (#26842) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-10-15 09:17:43 +00:00
ant-yy 5c3bae1a6a [Fix] Remove divisibility requirement between num_kv_heads and tp_size in bailing_moe (#26876) 
Signed-off-by: vito.yy <vito.yy@antgroup.com>
 2025-10-15 16:44:04 +08:00
Xudong MaandXudong Ma 5210dc3940 [Misc] Update TritonLanguagePlaceholder to have attributes that are used by Flash Linear Attention ops. (#26853) 
Co-authored-by: Xudong Ma <mxd@meta.com>
 2025-10-15 08:37:49 +00:00
Wentao Ye 71557a5f7c [CI] Fix mypy for vllm/executor (#26845) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-10-15 01:23:33 -07:00
Yongye Zhu f5ed68ef63 [Deepseek-V3.2][Kernel] Integrate cuda indexer k cache gather (#26456) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
 2025-10-15 16:05:01 +08:00
Angela Yi efdef57b1f [bugfix] Lazy import cv2 (#26869) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-10-15 07:47:50 +00:00
Mengqing Cao 302ef403a2 [DSA][MLA] Tiny refactor on DeepSeek to make it reusable for different backends (#26656) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-10-15 00:16:44 -07:00
sangho.lee 8865da157b [Bugfix][Multi Modal] Fix incorrect Molmo token processing (#26873) 
Signed-off-by: sanghol <sanghol@allenai.org>
 2025-10-15 07:13:59 +00:00
Isotr0py 8c851f6d04 [Bugfix] Fix qwen3-omni audio truncation issue (#26815) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-15 05:38:36 +00:00
Angela Yi 7cfa420f49 [BugFix] Patch inductor partitioning logic (#26735) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-10-15 05:04:32 +00:00