iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
3,443 Commits 3 Branches 0 Tags
4f168f69a3e856bda3f30e02fcee7db2a01ff32b
Commit Graph
969 Commits
Author SHA1 Message Date
youkaichao 3a763ba0c3 [core][misc] keep compatibility for old-style classes (#10356) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-15 13:55:51 +00:00
wchen61 3d158cdc8d Add default value to avoid Falcon crash (#5363) (#10347) 
Signed-off-by: wchen61 <wchen61@foxmail.com>
 2024-11-15 08:52:20 +00:00
Cyrus Leung 2ac6d0e75b [Misc] Consolidate pooler config overrides (#10351) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-15 06:59:00 +00:00
Cyrus Leung b40cf6402e [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) 2024-11-14 20:23:09 -08:00
Luka GovedičandDaniël de Kok bf2ddc6610 [bugfix] Fix static asymmetric quantization case (#10334) 
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
 2024-11-15 09:35:11 +08:00
Cyrus Leung 972112d82f [Bugfix] Fix unable to load some models (#10312) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-14 16:55:54 -08:00
Murali Andoorveedu b2e0ad3b59 [Perf] Reduce peak memory usage of llama (#10339) 
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
 2024-11-15 00:38:20 +00:00
Maximilien de BayserandFlavia Beo 4a18fd14ba Support Roberta embedding models (#9387) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
 2024-11-14 21:23:29 +00:00
B-201andJee Jee Li 294bf467ba [Model] Add BNB quantization support for Idefics3 (#10310) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-14 06:31:44 +00:00
youkaichao 504ac53d18 [misc] error early for old-style class (#10304) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-13 18:55:39 -08:00
Isotr0py 15bb8330aa [Bugfix] Fix tensor parallel for qwen2 classification model (#10297) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-14 10:54:59 +08:00
HoangCongDuc ac49b59d8b [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) 
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
 2024-11-13 09:56:39 -07:00
Cyrus Leung 0b8bb86bf1 [1/N] Initial prototype for multi-modal processor (#10044) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-13 12:39:03 +00:00
B-201 d909acf9fe [Model][LoRA]LoRA support added for idefics3 (#10281) 
Signed-off-by: B-201 <Joy25810@foxmail.com>
 2024-11-13 17:25:59 +08:00
Austin VeselkaandFurtherAI 1b886aa104 [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) 
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
 2024-11-13 08:28:13 +00:00
电脑星人 3945c82346 [Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-13 07:07:22 +00:00
Woosuk KwonandRoger Wang bbd3e86926 [V1] Support VLMs with fine-grained scheduling (#9871) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-11-13 04:53:13 +00:00
sroy745 b41fb9d3b1 [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) 
Signed-off-by: Sourashis Roy <sroy@roblox.com>
 2024-11-12 10:53:57 -08:00
Jie Fu (傅杰) 176fcb1c71 [Bugfix] Fix QwenModel argument (#10262) 
Signed-off-by: Jie Fu <jiefu@tencent.com>
 2024-11-12 16:36:51 +00:00
Jee Jee Li a838ba7254 [Misc]Fix Idefics3Model argument (#10255) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-12 13:07:11 +00:00
Isotr0py 58170d6503 [Hardware][CPU] Add embedding models support for CPU backend (#10193) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-11 08:54:28 +00:00
youkaichao f89d18ff74 [6/N] pass whole config to inner model (#10205) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-11 06:41:46 +00:00
Cyrus Leung 51c2e1fcef [CI/Build] Split up models tests (#10069) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-09 11:39:14 -08:00
youkaichao bd46357ad9 [bugfix] fix broken tests of mlp speculator (#10177) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-09 00:04:50 -08:00
youkaichao 1a95f10ee7 [5/N] pass the whole config to model (#9983) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-09 14:17:28 +08:00
Michael Goin f83feccd7f [Bugfix] Ignore GPTQ quantization of Qwen2-VL visual module (#10169) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-09 03:36:46 +00:00
Cyrus Leung e0191a95d8 [0/N] Rename MultiModalInputs to MultiModalKwargs (#10040) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-09 11:31:02 +08:00
rasmith 127c07480e [Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2024-11-08 19:59:22 -05:00
Cyrus Leung b489fc3c91 [CI/Build] Update CPU tests to include all "standard" tests (#5481) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-08 23:30:04 +08:00
Isotr0py 1ff4aed5bd [Model] Expose size to Idefics3 as mm_processor_kwargs (#10146) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-08 09:56:58 +00:00
litianjianlitianjianDarkLight1337
 		28b2877d30 Online video support for VLMs (#10020) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-07 20:25:59 +00:00
Jiahao Li 999df95b4e [Bugfix] Make image processor respect mm_processor_kwargs for Qwen2-VL (#10112) 
Signed-off-by: Jiahao Li <liplus17@163.com>
 2024-11-07 10:50:44 +00:00
Flávia BéoandMax de Bayser aa9078fa03 Adds method to read the pooling types from model's files (#9506) 
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2024-11-07 08:42:40 +00:00
Li, Jiang a4b3e0c1e9 [Hardware][CPU] Update torch 2.5 (#9911) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2024-11-07 04:43:08 +00:00
Michael Goin 4ab3256644 [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-07 00:54:13 +00:00
Joe Runde d58268c56a [V1] Make v1 more testable (#9888) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-11-06 11:57:35 -08:00
Michael Goin 399c798608 Remove ScaledActivation for AWQ (#10057) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-06 14:27:06 +00:00
Eric 406d4cc480 [Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) 
Signed-off-by: ericperfect <ericperfectttt@gmail.com>
 2024-11-06 14:13:15 +00:00
Jee Jee LiandB-201 a5bba7d234 [Model] Add Idefics3 support (#9767) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
 2024-11-06 11:41:17 +00:00
Jee Jee Li 2003cc3513 [Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-06 09:49:19 +00:00
+26
Konrad ZaworaKunshang JiSanju C SudhakaranMichal AdamczykMarceli FylcekHimangshu LahkarVivek GoelyuwenzhoDominika Olszewskabarak goldbergMichal SzutenbergJan KanieckiAgata DobrzyniewiczKrzysztof WisniewskiDudi LesterIlia TarabanChendi.XueMichał KuligowskiJakub MaksymczukTomasz ZielinskiSun ChoiIryna BoikoBob Zhuhlin99Zehao HuangAndrzej KotłowskiYan TomsinskyNir DavidYu-ZhouRuheena Suhani ShaikKarol DamaszkeMarcin SwiniarskiWoosuk KwonJacek CzajaJacek CzajaYuan
 		a02a50e6e5 [Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyniewicz-habana@users.noreply.github.com>
Co-authored-by: Krzysztof Wisniewski <kwisniewski@habana.ai>
Co-authored-by: Dudi Lester <160421192+dudilester@users.noreply.github.com>
Co-authored-by: Ilia Taraban <tarabanil@gmail.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Michał Kuligowski <mkuligowski@habana.ai>
Co-authored-by: Jakub Maksymczuk <jmaksymczuk@habana.ai>
Co-authored-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: Sun Choi <schoi@habana.ai>
Co-authored-by: Iryna Boiko <iboiko@habana.ai>
Co-authored-by: Bob Zhu <41610754+czhu15@users.noreply.github.com>
Co-authored-by: hlin99 <73271530+hlin99@users.noreply.github.com>
Co-authored-by: Zehao Huang <zehao.huang@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Yan Tomsinsky <73292515+Yantom1@users.noreply.github.com>
Co-authored-by: Nir David <ndavid@habana.ai>
Co-authored-by: Yu-Zhou <yu.zhou@intel.com>
Co-authored-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
Co-authored-by: Karol Damaszke <kdamaszke@habana.ai>
Co-authored-by: Marcin Swiniarski <mswiniarski@habana.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Jacek Czaja <jacek.czaja@intel.com>
Co-authored-by: Jacek Czaja <jczaja@habana.ai>
Co-authored-by: Yuan <yuan.zhou@outlook.com>
 2024-11-06 01:09:10 -08:00
Aaron Pham 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
arakowsk-amd ea928f608c [Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) 
Signed-off-by: Alex Rakowski <alex.rakowski@amd.com>
Signed-off-by: Alex Rakowski <182798202+arakowsk-amd@users.noreply.github.com>
 2024-11-06 05:10:40 +00:00
Peter Salas ffc0f2b47a [Model][OpenVINO] Fix regressions from #8346 (#10045) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-06 04:19:15 +00:00
Cyrus Leung 82bfc38d07 [Misc] Sort the list of embedding models (#10037) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-06 04:05:05 +00:00
zifeitong 43300bd98a [Bugfix] Properly propagate trust_remote_code settings (#10047) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
 2024-11-05 16:34:40 -08:00
Jee Jee Li b9c64c0ca7 [Misc] Modify BNB parameter name (#9997) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-05 14:40:08 -05:00
Michael Goin a53046b16f [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-05 10:42:20 -08:00
Tyler Michael Smith ad23318928 [Bugfix] Fixup Mamba (#10004) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-11-05 03:46:38 +00:00
Cyrus Leung bbc3619dc8 [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-05 10:07:31 +08:00