iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,386 Commits 3 Branches 0 Tags
fa6a6be51978bd4b49ba0da17039e60f96dc5b13
Commit Graph
2340 Commits
Author SHA1 Message Date
Ayush SatyamandDarkLight1337 5f7e8a916a [Model] Define merge_by_field_config MM interface (U-Z) (#26261) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 06:45:49 +00:00
Benjamin Chislett 2161efe978 [Bugfix] Allow skipping MoE in NVFP4 (fix for MTP) (#25987) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-10-06 16:16:30 -04:00
Raushan TurganbayandHarry Mellor 7cd95dc8a3 [Bugfix] Fix gemma3 with transformers backend (#23178) 
Signed-off-by: raushan <raushan@huggingface.co>
Signed-off-by: Raushan Turganbay <raushan@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 18:42:32 +00:00
Rahul Tuli 05f6846ede Support llama3 eagle3 head with llama4 verifier (#25961) 
Signed-off-by: rahul-tuli <rtuli@redhat.com>
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
 2025-10-06 13:56:08 -04:00
tomeras91gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Eugene Khvedchenia
 		b8f603cebe [Model] EVS support for nano_nemotron_vl (#26269) 
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
 2025-10-07 00:23:37 +08:00
Chatcharin Sangbutsarakum fc679696f8 Fix DotsOCR tensor type (#26281) 
Signed-off-by: what_in_the_nim <chatcharinsang@gmail.com>
 2025-10-06 12:23:43 +00:00
Raushan TurganbayandHarry Mellor ab5e7d93f4 [Bugfix] Fix mrope in Transformers Backend (#26087) 
Signed-off-by: raushan <raushan@huggingface.co>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 11:40:50 +00:00
Harry Mellor 0340f45553 Support expert parallel load balancing in Transformers backend (#26287) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 11:20:16 +00:00
Cyrus Leung 19a00eb210 [Model] Use merge_by_field_config for MM models (Llava family) (#26280) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-06 09:45:26 +00:00
Roger Wang 43c146ca42 [Misc] Clean up unnecessary E501 ignore (#26274) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-10-06 07:29:18 +00:00
Harry Mellor 6c04638214 Fix per file ruff ignores related to line length (#26262) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-06 05:12:40 +00:00
Harry Mellor 557b2e961d Remove all cases of fmt: on/off (#26253) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:14 -07:00
Harry Mellor 4e256cadc2 Remove all references to yapf as it's no longer used (#26251) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 09:18:11 -07:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Cyrus Leung b7e8e4e6be [Bugfix] Always apply MM processor even when no MM items are passed (#26240) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-05 10:10:20 +00:00
Cyrus Leung 59a85c366e [Model] Use merge_by_field_config for MM models (H-L) (#26230) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-05 11:54:17 +08:00
Thomas Parnell ed3aeb25a4 [V1] [Hybrid] Remove code to override default CUDA graph configuration (#26226) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-04 13:47:48 +00:00
yuafngYuanfeng LiCyrus Leung
 		86ee949128 Fix tensor device and dtype placement in Qwen2VL model (#26219) 
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Yuanfeng Li <yuanfengli@meta.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-04 06:41:39 -07:00
Cyrus Leung 4570535ec4 [Model] CLIP Embedding Support (#26010) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-04 06:21:42 -07:00
Cyrus Leung 44ea85137a [Model] Support nested structures for TensorSchema (#26212) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-04 01:20:32 -07:00
Harry MellorandIsotr0py d3d649efec Support expert parallel in Transformers backend (#26162) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-10-04 04:35:04 +00:00
Stan WozniakThomas OrtnerThomas Parnell
 		ea507c3a93 [V1] [Hybrid] Mamba2 Automatic Prefix Caching (#25752) 
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Ortner <boh@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-10-04 06:34:22 +02:00
Roger Wang 67bc0c003e [Bugfix] Fix qwen3 vl dummy data generation with overrides (#26193) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-10-04 01:40:20 +00:00
Eugene Khvedchenya 5a05f26603 Fix issue of using only the part of video frame [Nemotron Nano] (#26186) 
Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
 2025-10-04 00:21:00 +00:00
Chendi.XueandRoger Wang dd96465fd7 [BugFix][QWEN-VL]fix wrong apply_rotary_emb_torch selection introduced by #24642 (#26123) 
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-10-03 08:52:26 -07:00
whx cbf9221992 [Model] Supplement to PR 24862: Pass param prefix to LLMHead (#25805) 
Signed-off-by: whx-sjtu <2952154980@qq.com>
 2025-10-03 21:34:53 +08:00
Wenlong WangandHarry Mellor 79aa244678 [Multi Modal] Configurable MM Profiling (#25631) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-03 03:59:10 -07:00
Cyrus Leung f9a8084e48 [Model] Use merge_by_field_config for MM models (InternVL family) (#26153) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-03 01:59:06 -07:00
Cyrus Leung 0ad9951c41 [Input] Remove unused prompt field (#26097) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-03 00:23:21 -07:00
Harry Mellor 10d765482d FusedMoE support for the Transformers backend (#22650) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-02 23:12:15 -07:00
Cyrus Leung 39b643dc1a [Model] Use merge_by_field_config for MM models (G) (#26117) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-02 22:38:29 -07:00
TJian 9c5ee91b2a [ROCm] [VL] [Bugfix] Fix vit flash attn dispatcher logic for ROCm (#26104) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-10-02 22:34:53 -07:00
Matthew Bonanni 2aaa423842 [Attention] Move Backend enum into registry (#25893) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-10-02 20:32:24 -07:00
Chen Zhang 1e50f1be70 [Deepseek v3.2] Support indexer prefill chunking (#25999) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-10-02 10:29:12 -07:00
vllmellm 5e4a8223c6 [Qwen][ROCm] Flash Attention Rotary Embeddings (#24642) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-10-02 08:26:08 -07:00
Cyrus Leung cc253b73d3 [Model] Use merge_by_field_config for MM models (D-F) (#26076) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-02 08:17:35 -07:00
Cyrus Leung 7d6fb905d9 [Model] Use merge_by_field_config for MM models (A-C) (#26073) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-02 08:17:31 -07:00
Cyrus Leung 1405f0c7ba [Misc] Factor out common _apply_feature_select_strategy (#26003) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-01 01:31:03 -07:00
Wenlong Wang 84d57342b6 [BugFix][MM] Fix Nonetype error when video is cache in qwen2.5-omni-thinker (#26004) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-10-01 08:03:25 +00:00
Harry MellorCyrus LeungCyrus LeungIsotr0py
 		2a69ab4899 Update to Transformers v4.56.2 (#24638) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-30 22:07:07 -07:00
Lucas Wilkinson 8d7da92fd7 [BugFix] Fix default kv-cache-dtype default for DeepseekV3.2 (#25988) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-09-30 21:58:31 -07:00
Roger Wang 66bca9b8bd [MM] Add text-only mode for Qwen3-VL (#26000) 2025-09-30 21:13:42 -07:00
Harry MellorandIsotr0py a388252ac4 Add explicit pooling classes for the Transformers backend (#25322) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-09-30 23:07:06 +01:00
Cyrus Leung 9f1c4ecaf2 [Bugfix] Token type and position embeddings fail to be applied to inputs_embeds (#25922) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-01 00:23:12 +08:00
Anion f4db5e6de1 [Bugfix][Model] Fix inference for Hunyuan dense models (#25354) 
Signed-off-by: anion <1005128408@qq.com>
Signed-off-by: Anion <123177548+Anionex@users.noreply.github.com>
 2025-09-30 14:38:07 +00:00
Cyrus Leung d7e34b4210 [Model] Move vision_feature_select_strategy into resolve_visual_encoder_outputs (#25938) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-09-30 11:24:57 +00:00
CSWYF3634076 ef6e0e7132 [Bugfix][Model]fix ernie45 moe gate&bias dtype to float32 (#25936) 
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
 2025-09-30 19:11:21 +08:00
+6
Yongye ZhuChen ZhangyoukaichaoLucas WilkinsonRobert ShawLucas Wilkinsonyewentao256Wentao YemgoinLucia FangLucia FangNickLuccheSiyuan FuMatthew BonanniXiaozhu MengBarry Kang
 		fa7e254a7f [New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
 2025-09-30 17:14:41 +08:00
Zhou Jiahao 2e1b8bc2b6 [Model][Bugfix] Fix MiDashengLM audio encoder mask by removing incorrect logical_not (#25925) 
Signed-off-by: zhoukz <me@zhoukz.com>
 2025-09-30 08:15:23 +00:00
Harry Mellor 61aedb5ffe MoveVllmConfig from config/__init__.py to config/vllm.py (#25271) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-29 19:49:49 -07:00