iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
14,683 Commits 3 Branches 0 Tags
42fadebecb79290ad722f33f3094de23b121f33d
Commit Graph
4305 Commits
Author SHA1 Message Date
Austin VeselkaandFurtherAI 1b886aa104 [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) 
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
 2024-11-13 08:28:13 +00:00
电脑星人 3945c82346 [Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221) 
Signed-off-by: imkero <kerorek@outlook.com>
 2024-11-13 07:07:22 +00:00
Woosuk KwonandRoger Wang bbd3e86926 [V1] Support VLMs with fine-grained scheduling (#9871) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-11-13 04:53:13 +00:00
sroy745 b41fb9d3b1 [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) 
Signed-off-by: Sourashis Roy <sroy@roblox.com>
 2024-11-12 10:53:57 -08:00
Jie Fu (傅杰) 176fcb1c71 [Bugfix] Fix QwenModel argument (#10262) 
Signed-off-by: Jie Fu <jiefu@tencent.com>
 2024-11-12 16:36:51 +00:00
Jee Jee Li a838ba7254 [Misc]Fix Idefics3Model argument (#10255) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-12 13:07:11 +00:00
Isotr0py 58170d6503 [Hardware][CPU] Add embedding models support for CPU backend (#10193) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-11 08:54:28 +00:00
youkaichao f89d18ff74 [6/N] pass whole config to inner model (#10205) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-11 06:41:46 +00:00
Cyrus Leung 51c2e1fcef [CI/Build] Split up models tests (#10069) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-09 11:39:14 -08:00
youkaichao bd46357ad9 [bugfix] fix broken tests of mlp speculator (#10177) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-09 00:04:50 -08:00
youkaichao 1a95f10ee7 [5/N] pass the whole config to model (#9983) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-09 14:17:28 +08:00
Michael Goin f83feccd7f [Bugfix] Ignore GPTQ quantization of Qwen2-VL visual module (#10169) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-09 03:36:46 +00:00
Cyrus Leung e0191a95d8 [0/N] Rename MultiModalInputs to MultiModalKwargs (#10040) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-09 11:31:02 +08:00
rasmith 127c07480e [Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
 2024-11-08 19:59:22 -05:00
Cyrus Leung b489fc3c91 [CI/Build] Update CPU tests to include all "standard" tests (#5481) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-08 23:30:04 +08:00
Isotr0py 1ff4aed5bd [Model] Expose size to Idefics3 as mm_processor_kwargs (#10146) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2024-11-08 09:56:58 +00:00
litianjianlitianjianDarkLight1337
 		28b2877d30 Online video support for VLMs (#10020) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-07 20:25:59 +00:00
Jiahao Li 999df95b4e [Bugfix] Make image processor respect mm_processor_kwargs for Qwen2-VL (#10112) 
Signed-off-by: Jiahao Li <liplus17@163.com>
 2024-11-07 10:50:44 +00:00
Flávia BéoandMax de Bayser aa9078fa03 Adds method to read the pooling types from model's files (#9506) 
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2024-11-07 08:42:40 +00:00
Li, Jiang a4b3e0c1e9 [Hardware][CPU] Update torch 2.5 (#9911) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2024-11-07 04:43:08 +00:00
Michael Goin 4ab3256644 [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-07 00:54:13 +00:00
Joe Runde d58268c56a [V1] Make v1 more testable (#9888) 
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
 2024-11-06 11:57:35 -08:00
Michael Goin 399c798608 Remove ScaledActivation for AWQ (#10057) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-06 14:27:06 +00:00
Eric 406d4cc480 [Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) 
Signed-off-by: ericperfect <ericperfectttt@gmail.com>
 2024-11-06 14:13:15 +00:00
Jee Jee LiandB-201 a5bba7d234 [Model] Add Idefics3 support (#9767) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
 2024-11-06 11:41:17 +00:00
Jee Jee Li 2003cc3513 [Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-06 09:49:19 +00:00
+26
Konrad ZaworaKunshang JiSanju C SudhakaranMichal AdamczykMarceli FylcekHimangshu LahkarVivek GoelyuwenzhoDominika Olszewskabarak goldbergMichal SzutenbergJan KanieckiAgata DobrzyniewiczKrzysztof WisniewskiDudi LesterIlia TarabanChendi.XueMichał KuligowskiJakub MaksymczukTomasz ZielinskiSun ChoiIryna BoikoBob Zhuhlin99Zehao HuangAndrzej KotłowskiYan TomsinskyNir DavidYu-ZhouRuheena Suhani ShaikKarol DamaszkeMarcin SwiniarskiWoosuk KwonJacek CzajaJacek CzajaYuan
 		a02a50e6e5 [Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) 
Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzyniewicz <160237065+adobrzyniewicz-habana@users.noreply.github.com>
Co-authored-by: Krzysztof Wisniewski <kwisniewski@habana.ai>
Co-authored-by: Dudi Lester <160421192+dudilester@users.noreply.github.com>
Co-authored-by: Ilia Taraban <tarabanil@gmail.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Michał Kuligowski <mkuligowski@habana.ai>
Co-authored-by: Jakub Maksymczuk <jmaksymczuk@habana.ai>
Co-authored-by: Tomasz Zielinski <85164140+tzielinski-habana@users.noreply.github.com>
Co-authored-by: Sun Choi <schoi@habana.ai>
Co-authored-by: Iryna Boiko <iboiko@habana.ai>
Co-authored-by: Bob Zhu <41610754+czhu15@users.noreply.github.com>
Co-authored-by: hlin99 <73271530+hlin99@users.noreply.github.com>
Co-authored-by: Zehao Huang <zehao.huang@intel.com>
Co-authored-by: Andrzej Kotłowski <Andrzej.Kotlowski@intel.com>
Co-authored-by: Yan Tomsinsky <73292515+Yantom1@users.noreply.github.com>
Co-authored-by: Nir David <ndavid@habana.ai>
Co-authored-by: Yu-Zhou <yu.zhou@intel.com>
Co-authored-by: Ruheena Suhani Shaik <rsshaik@habana.ai>
Co-authored-by: Karol Damaszke <kdamaszke@habana.ai>
Co-authored-by: Marcin Swiniarski <mswiniarski@habana.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Jacek Czaja <jacek.czaja@intel.com>
Co-authored-by: Jacek Czaja <jczaja@habana.ai>
Co-authored-by: Yuan <yuan.zhou@outlook.com>
 2024-11-06 01:09:10 -08:00
Aaron Pham 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
arakowsk-amd ea928f608c [Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) 
Signed-off-by: Alex Rakowski <alex.rakowski@amd.com>
Signed-off-by: Alex Rakowski <182798202+arakowsk-amd@users.noreply.github.com>
 2024-11-06 05:10:40 +00:00
Peter Salas ffc0f2b47a [Model][OpenVINO] Fix regressions from #8346 (#10045) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-06 04:19:15 +00:00
Cyrus Leung 82bfc38d07 [Misc] Sort the list of embedding models (#10037) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-06 04:05:05 +00:00
zifeitong 43300bd98a [Bugfix] Properly propagate trust_remote_code settings (#10047) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
 2024-11-05 16:34:40 -08:00
Jee Jee Li b9c64c0ca7 [Misc] Modify BNB parameter name (#9997) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-05 14:40:08 -05:00
Michael Goin a53046b16f [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-05 10:42:20 -08:00
Tyler Michael Smith ad23318928 [Bugfix] Fixup Mamba (#10004) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-11-05 03:46:38 +00:00
Cyrus Leung bbc3619dc8 [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-05 10:07:31 +08:00
Michael Goin 8f0a9ca890 [Bugfix] Respect modules_to_not_convert within awq_marlin (#9895) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-04 16:57:44 -07:00
Mor Zusman 603a661ae8 [Model] factoring out MambaMixer out of Jamba (#8993) 
Signed-off-by: mzusman <mor.zusmann@gmail.com>
 2024-11-04 18:00:00 +00:00
Jee Jee Li fb2716d641 [Misc]Reduce BNB static variable (#9987) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-04 17:04:40 +00:00
youkaichao 8d72bb20fa [4/N] make quant config first-class citizen (#9978) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-04 08:51:31 -08:00
Jee Jee Li c49f0407ba [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-04 03:36:41 +00:00
shanshan wangandRoger Wang 54597724f4 [Model] Add support for H2OVL-Mississippi models (#9747) 
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-11-04 00:15:36 +00:00
youkaichao 3bb4befea7 [bugfix] fix tsts (#9959) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 15:54:05 -07:00
Yongzao ae5279a163 [torch.compile] Adding torch compile to vision-language models (#9946) 2024-11-02 12:56:05 -07:00
youkaichao cea808f325 [3/N] model runner pass the whole config to model (#9958) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 12:08:49 -07:00
youkaichao 74b529ceee [bugfix] fix chatglm dummy_data_for_glmv (#9955) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 08:03:33 -07:00
sroy745 a78dd3303e [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) 2024-11-01 23:22:49 -07:00
Peter Salas 6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-01 16:21:10 -07:00
Pavani Majety 598b6d7b07 [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) 2024-11-01 12:15:05 -07:00
Yongzaoandyoukaichao 2b5bf20988 [torch.compile] Adding torch compile annotations to some models (#9876) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-11-01 00:25:47 -07:00