iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
3,287 Commits 3 Branches 0 Tags
098f94de42859f8251fe920f87adb88336129c53
Commit Graph
494 Commits
Author SHA1 Message Date
Michael Goin 399c798608 Remove ScaledActivation for AWQ (#10057) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-06 14:27:06 +00:00
Eric 406d4cc480 [Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (#10022) 
Signed-off-by: ericperfect <ericperfectttt@gmail.com>
 2024-11-06 14:13:15 +00:00
Jee Jee LiandB-201 a5bba7d234 [Model] Add Idefics3 support (#9767) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
 2024-11-06 11:41:17 +00:00
Jee Jee Li 2003cc3513 [Model][LoRA]LoRA support added for LlamaEmbeddingModel (#10071) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-06 09:49:19 +00:00
Aaron Pham 21063c11c7 [CI/Build] drop support for Python 3.8 EOL (#8464) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2024-11-06 07:11:55 +00:00
arakowsk-amd ea928f608c [Bugfix] Gpt-j-6B patch kv_scale to k_scale path (#10063) 
Signed-off-by: Alex Rakowski <alex.rakowski@amd.com>
Signed-off-by: Alex Rakowski <182798202+arakowsk-amd@users.noreply.github.com>
 2024-11-06 05:10:40 +00:00
Peter Salas ffc0f2b47a [Model][OpenVINO] Fix regressions from #8346 (#10045) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-06 04:19:15 +00:00
Cyrus Leung 82bfc38d07 [Misc] Sort the list of embedding models (#10037) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-06 04:05:05 +00:00
zifeitong 43300bd98a [Bugfix] Properly propagate trust_remote_code settings (#10047) 
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
 2024-11-05 16:34:40 -08:00
Michael Goin a53046b16f [Model] Support quantization of PixtralHFTransformer for PixtralHF (#9921) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-05 10:42:20 -08:00
Tyler Michael Smith ad23318928 [Bugfix] Fixup Mamba (#10004) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2024-11-05 03:46:38 +00:00
Cyrus Leung bbc3619dc8 [Core] Make encoder-decoder inputs a nested structure to be more composable (#9604) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-05 10:07:31 +08:00
Mor Zusman 603a661ae8 [Model] factoring out MambaMixer out of Jamba (#8993) 
Signed-off-by: mzusman <mor.zusmann@gmail.com>
 2024-11-04 18:00:00 +00:00
Jee Jee Li fb2716d641 [Misc]Reduce BNB static variable (#9987) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-04 17:04:40 +00:00
Jee Jee Li c49f0407ba [Bugfix] Fix MiniCPMV and Mllama BNB bug (#9917) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-11-04 03:36:41 +00:00
shanshan wangandRoger Wang 54597724f4 [Model] Add support for H2OVL-Mississippi models (#9747) 
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2024-11-04 00:15:36 +00:00
youkaichao 3bb4befea7 [bugfix] fix tsts (#9959) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 15:54:05 -07:00
Yongzao ae5279a163 [torch.compile] Adding torch compile to vision-language models (#9946) 2024-11-02 12:56:05 -07:00
youkaichao 74b529ceee [bugfix] fix chatglm dummy_data_for_glmv (#9955) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-11-02 08:03:33 -07:00
sroy745 a78dd3303e [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) 2024-11-01 23:22:49 -07:00
Peter Salas 6c0b7f548d [Core][VLM] Add precise multi-modal placeholder tracking (#8346) 
Signed-off-by: Peter Salas <peter@fixie.ai>
 2024-11-01 16:21:10 -07:00
Yongzaoandyoukaichao 2b5bf20988 [torch.compile] Adding torch compile annotations to some models (#9876) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-11-01 00:25:47 -07:00
Michael Goin 93a76dd21d [Model] Support bitsandbytes for MiniCPMV (#9891) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-11-01 13:31:56 +08:00
youkaichao 566cd27797 [torch.compile] rework test plans (#9866) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-31 22:20:17 -07:00
Jee Jee Li 5608e611c2 [Doc] Update Qwen documentation (#9869) 2024-10-31 08:54:18 +00:00
Michael Goin d087bf863e [Model] Support quantization of Qwen2VisionTransformer (#9817) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-10-30 22:41:20 -07:00
Went-Liang 81f09cfd80 [Model] Support math-shepherd-mistral-7b-prm model (#9697) 
Signed-off-by: Went-Liang <wenteng_liang@163.com>
 2024-10-30 09:33:42 -07:00
Michael Goin bc73e9821c [Bugfix] Fix prefix strings for quantized VLMs (#9772) 2024-10-29 16:02:59 -07:00
Junichi Sato ac3d748dba [Model] Add LlamaEmbeddingModel as an embedding Implementation of LlamaModel (#9806) 2024-10-29 10:40:35 -07:00
Isotr0py 09500f7dde [Model] Add BNB quantization support for Mllama (#9720) 2024-10-29 08:20:02 -04:00
Cyrus Leung eae3d48181 [Bugfix] Use temporary directory in registry (#9721) 2024-10-28 22:08:20 -07:00
Jee Jee Li 7a4df5f200 [Model][LoRA]LoRA support added for Qwen (#9622) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2024-10-29 04:14:07 +00:00
youkaichao 76ed5340f0 [torch.compile] add deepseek v2 compile (#9775) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-28 14:35:17 -07:00
Yongzao aa0addb397 Adding "torch compile" annotations to moe models (#9758) 2024-10-28 13:49:56 -07:00
litianjianlitianjianDarkLight1337
 		5f8d8075f9 [Model][VLM] Add multi-video support for LLaVA-Onevision (#8905) 
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-28 18:04:10 +00:00
youkaichao 32176fee73 [torch.compile] support moe models (#9632) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-10-27 21:58:04 -07:00
wangshuai09 4e2d95e372 [Hardware][ROCM] using current_platform.is_rocm (#9642) 
Signed-off-by: wangshuai09 <391746016@qq.com>
 2024-10-28 04:07:00 +00:00
kakao-kevin-usandKevin-Yang 6650e6a930 [Model] Add classification Task with Qwen2ForSequenceClassification (#9704) 
Signed-off-by: Kevin-Yang <ykcha9@gmail.com>
Co-authored-by: Kevin-Yang <ykcha9@gmail.com>
 2024-10-26 17:53:35 +00:00
Will Johnson a6f3721861 [Model] add a lora module for granite 3.0 MoE models (#9673) 2024-10-24 22:00:17 -07:00
Michael Goin c91ed47c43 [Bugfix] Remove xformers requirement for Pixtral (#9597) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-10-24 15:38:05 -07:00
Alex Brooks 722d46edb9 [Model] Compute Llava Next Max Tokens / Dummy Data From Gridpoints (#9650) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2024-10-24 10:42:24 -07:00
Yongzaoandyoukaichao d27cfbf791 [torch.compile] Adding torch compile annotations to some models (#9641) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-10-24 09:31:42 -07:00
litianjian f58454968f [Bugfix]Disable the post_norm layer of the vision encoder for LLaVA models (#9653) 2024-10-24 07:52:07 -07:00
Yongzaoandyoukaichao ad6f78053e [torch.compile] expanding support and fix allgather compilation (#9637) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-10-24 01:32:15 -07:00
Yongzaoandyoukaichao 8a02cd045a [torch.compile] Adding torch compile annotations to some models (#9639) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
 2024-10-24 00:54:57 -07:00
Cyrus Leung 836e8ef6ee [Bugfix] Fix PP for ChatGLM and Molmo (#9422) 2024-10-24 06:12:05 +00:00
Michael Goin b7df53cd42 [Bugfix] Use "vision_model" prefix for MllamaVisionModel (#9628) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-10-24 10:07:44 +08:00
Michael Goin bb01f2915e [Bugfix][Model] Fix Mllama SDPA illegal memory access for batched multi-image (#9626) 
Signed-off-by: mgoin <michael@neuralmagic.com>
 2024-10-24 10:03:44 +08:00
Yunfei ChuandDarkLight1337 fc6c274626 [Model] Add Qwen2-Audio model support (#9248) 
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-10-23 17:54:22 +00:00
Yongzao 9013e24f7b [torch.compile] Adding torch compile annotations to some models (#9614) 2024-10-23 10:07:48 -07:00