iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
5,069 Commits 3 Branches 0 Tags
4290b704ffe025de8efdd2ecce062ec8bc34e1d6
Commit Graph
805 Commits
Author SHA1 Message Date
ChaunceyandDarkLight1337 92b0ce2ac7 [Bugfix][v1] fixed llava-hf/llava-1.5-7b-hf is broken on V1 (#14554) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-10 18:24:51 +00:00
Harry Mellor 3b352a2f92 Correct capitalisation: VLLM -> vLLM (#14562) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-10 16:36:21 +00:00
Isotr0py fb16eea48b [Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB quantization work (#14498) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-09 04:47:45 +00:00
Isotr0py 03fe18ae0f [VLM] Add TP support for Phi-4-MM (#14453) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-08 05:57:14 -08:00
Jee Jee Li 952a074980 [Misc] Add Phi4-MM example (#14343) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-07 17:28:52 +00:00
Cyrus Leung 05fb6718f0 [Bugfix] Clean up multi-modal processors (#14417) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-03-07 10:33:38 +00:00
Nicolò LucchesiandNickLucche 69ff99fdcd [Core] Optimizing cross-attention QKVParallelLinear computation (#12325) 
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
 2025-03-06 09:37:26 +00:00
lkchen 5d802522a7 [V1][VLM][Pixtral-HF] Support Pixtral-HF on V1 (#14275) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-03-06 08:58:41 +00:00
kYLeandCyrus Leung 1769928079 [Model] Update Paligemma multimodal processing with PromptUpdate (#14015) 
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-03-06 08:31:38 +00:00
Isotr0py e17e4488bd [LoRA] Remove linear hack outside transformers backend (#14177) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-03-05 15:06:28 +00:00
Roger Wang ec79b67c77 [Misc][V1] Avoid using envs.VLLM_USE_V1 in mm processing (#14256) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-03-05 07:37:16 +00:00
Tyler Michael Smith 72c62eae5f [V1] EP/TP MoE + DP Attention (#13931) 2025-03-04 21:27:26 -08:00
Congcong Chen 0a995d5434 [Model] New model support for Phi-4-multimodal-instruct (#14119) 2025-03-04 20:57:01 -08:00
Tyler Michael Smith 4f5b059f14 Clean up unused padding_idx variables across many model definitions (#13240) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-03-04 21:27:00 +00:00
lkchen b3cf368d79 [V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161) 2025-03-04 15:43:59 +00:00
Travis JohnsonandCyrus Leung c060b71408 [Model] Add support for GraniteMoeShared models (#13313) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-03-04 08:04:52 +08:00
Michael Goin 2b04c209ee [Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-03-03 14:20:24 -07:00
Harry Mellor 91373a0d15 Fix head_dim not existing in all model configs (Transformers backend) (#14141) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-03-03 17:48:11 +00:00
wang.yuqi e584b85afd [Misc] duplicate code in deepseek_v2 (#14106) 2025-03-03 14:10:11 +08:00
Jee Jee Li cc5e8f6db8 [Model] Add LoRA support for TransformersModel (#13770) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-03-02 09:17:34 +08:00
Isotr0py fdcc405346 [Doc] Consolidate whisper and florence2 examples (#14050) 2025-02-28 22:49:15 -08:00
Yang Liu 9b61dd41e7 [Bugfix] Initialize attention bias on the same device as Query/Key/Value for QwenVL Series (#14031) 2025-02-28 07:36:08 -08:00
Cyrus Leung f7bee5c815 [VLM][Bugfix] Enable specifying prompt target via index (#14038) 2025-02-28 07:35:55 -08:00
Cyrus Leung 1088f06242 [Doc] Move multimodal Embedding API example to Online Serving page (#14017) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-28 07:12:04 +00:00
Travis Johnson 73e0225ee9 [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#13911) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2025-02-28 04:00:45 +00:00
Roger Wang 6c85da3a18 [V1]SupportsV0Only protocol for model definitions (#13959) 
Signed-off-by: Roger Wang <ywang@roblox.com>
 2025-02-27 20:02:15 -05:00
Benjamin Chislett 9804145cac [Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
 2025-02-27 15:28:08 -08:00
Yang Chen 58d1b2aa77 [Attention] MLA support for V1 (#13789) 
Signed-off-by: Yang Chen <yangche@fb.com>
 2025-02-27 13:14:17 -05:00
Cyrus Leung f1579b229d [VLM] Generalized prompt updates for multi-modal processor (#13964) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-27 17:44:25 +00:00
Isotr0py 7864875879 [Bugfix] Fix qwen2.5-vl overflow issue (#13968) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-02-27 17:30:39 +00:00
Isotr0py edf309ebbe [VLM] Support multimodal inputs for Florence-2 models (#13320) 2025-02-27 02:06:41 -08:00
Roger Wang 7ca1da020f [Misc] Fix input processing for Ultravox (#13871) 2025-02-25 23:56:34 -08:00
Harry Mellor 24679788ed DeepSeek V2/V3/R1 only place lm_head on last pp rank (#13833) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-02-26 01:24:57 +00:00
Michael Goin 07c4353057 [Model] Support Grok1 (#13795) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-02-26 01:07:12 +00:00
Cyrus Leung f4133ce4e5 [Bugfix] Revert inspection code in #13743 (#13832) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-26 00:18:50 +08:00
Isotr0py 6ff518626c [Bugfix] Fix deepseek-vl2 inference with more than 2 images (#13818) 2025-02-25 06:03:02 -08:00
Cyrus Leung 6724e79164 [Misc] Check that the model can be inspected upon registration (#13743) 2025-02-25 00:18:19 -08:00
Lucas Wilkinson 4a8cfc7551 [Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" (#13802) 2025-02-24 20:33:59 -08:00
cjackal 51010a1807 [Misc] set single whitespace between log sentences (#13771) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-02-25 10:26:12 +08:00
Harry Mellor cdc1fa12eb Remove unused kwargs from model definitions (#13555) 2025-02-24 17:13:52 -08:00
Jongseok Park 781096e385 Expert Parallelism (EP) Support for DeepSeek V2 (#12583) 2025-02-24 07:33:20 -08:00
Mengqing Cao 23eca9cf68 [model][refactor] remove cuda hard code in models and layers (#13658) 2025-02-24 06:10:14 -08:00
Isotr0py ba5106e519 [LMM] Implement merged multimodal processor for whisper (#13278) 2025-02-23 01:46:03 -08:00
Kyle Sayers d5ca2110f1 [Quant] BaiChuan SupportsQuant (#13710) 2025-02-22 19:21:15 -08:00
Jee Jee Li 105b8ce4c0 [Misc] Reduce LoRA-related static variable (#13166) 2025-02-22 00:21:30 -08:00
Shane A 9a1f1da5d1 [Bugfix][Model] OLMo 2: split qkv correctly for GQA and MQA (#13687) 2025-02-21 22:07:45 -08:00
燃 041e294716 [Misc] add mm_processor_kwargs to extra_body for Qwen2.5-VL (#13533) 2025-02-19 23:04:30 -08:00
Jee Jee Li 512368e34a [Misc] Qwen2.5 VL support LoRA (#13261) 2025-02-19 18:37:55 -08:00
Cyrus Leung 377d10bd14 [VLM][Bugfix] Pass processor kwargs properly on init (#13516) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-02-19 13:13:50 +00:00
Lucia FangandLiuXiaoxuanPKU f525c0be8b [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) 
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
 2025-02-19 17:06:23 +08:00