Kunshang Ji and Jiang Li
96b6f475dd
Remove hardcoded device="cuda" to support more devices ( #2503 )
...
Co-authored-by: Jiang Li <jiang1.li@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2024-02-01 15:46:39 -08:00
Pernekhan Utemuratov and Pernekhan Utemuratov
c410f5d020
Use revision when downloading the quantization config file ( #2697 )
...
Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com >
2024-02-01 15:41:58 -08:00
Woosuk Kwon
3dad944485
Add quantized mixtral support ( #2673 )
2024-01-30 16:34:10 -08:00
9b945daaf1
[Experimental] Add multi-LoRA support ( #1804 )
...
Co-authored-by: Chen Shen <scv119@gmail.com >
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com >
Co-authored-by: Avnish Narayan <avnish@anyscale.com >
2024-01-23 15:26:37 -08:00
Woosuk Kwon
518369d78c
Implement lazy model loader ( #2044 )
2023-12-12 22:21:45 -08:00
b5f882cc98
Mixtral 8x7B support ( #2011 )
...
Co-authored-by: Pierre Stock <p@mistral.ai >
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com >
2023-12-11 01:09:15 -08:00
6ccc0bfffb
Merge EmbeddedLLM/vllm-rocm into vLLM main ( #1836 )
...
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com >
Co-authored-by: Amir Balwel <amoooori04@gmail.com >
Co-authored-by: root <kuanfu.liu@akirakan.com >
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com >
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com >
2023-12-07 23:16:52 -08:00
dancingpipi and shujunhua1
1d9b737e05
Support ChatGLMForConditionalGeneration ( #1932 )
...
Co-authored-by: shujunhua1 <shujunhua1@jd.com >
2023-12-05 10:52:48 -08:00
ljss
a8b150c595
Init model on GPU to reduce CPU memory footprint ( #1796 )
2023-11-27 11:18:26 -08:00
Simon Mo
5ffc0d13a2
Migrate linter from pylint to ruff ( #1665 )
2023-11-20 11:58:01 -08:00
Woosuk Kwon
bb00f66e19
Use quantization_config in hf config ( #1695 )
2023-11-17 16:23:49 -08:00
maximzubkov
521b35f799
Support Microsoft Phi 1.5 ( #1664 )
2023-11-16 14:28:39 -08:00
Megha Agarwal
b514d3c496
Revert MptConfig to MPTConfig ( #1668 )
2023-11-16 01:19:39 -08:00
Zhuohan Li
7076fa1c9f
TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models ( #1622 )
...
Refactor the tensor parallelism, quantization, and weight-loading codes.
Summary of the new features enabled by this PR:
- **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580 ).
- Model loading code became much simpler.
- Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.
2023-11-15 22:50:41 -08:00
forpanyang
ab9e8488d5
Add Yi model to quantization support ( #1600 )
2023-11-09 11:47:14 -08:00
GoHomeToMacDonal
1a2bbc9301
ChatGLM Support ( #1261 )
2023-11-06 16:09:33 -08:00
Roy
e7f579eb97
Support Yi model ( #1567 )
2023-11-06 15:26:03 -08:00
Woosuk Kwon
1fe0990023
Remove MPTConfig ( #1529 )
2023-11-01 15:29:05 -07:00
Wenfei Yan
cf8849f2d6
Add MptForCausalLM key in model_loader ( #1526 )
2023-10-31 15:46:53 -07:00
Lu Wang
de89472897
Fix the issue for AquilaChat2-* models ( #1339 )
2023-10-13 11:51:29 -07:00
amaleshvemula
ee8217e5be
Add Mistral to quantization model list ( #1278 )
2023-10-11 00:26:24 -07:00
Chris Bamford and timlacroix
bb1ba58f06
[Mistral] Mistral-7B-v0.1 support ( #1196 )
...
Co-authored-by: timlacroix <t@mistral.ai >
2023-09-28 10:41:03 -07:00
Woosuk Kwon
2b1c116b5a
Add minimum capability requirement for AWQ ( #1064 )
2023-09-18 12:02:01 -07:00
e3e79e9e8a
Implement AWQ quantization support for LLaMA ( #1032 )
...
Co-authored-by: Robert Irvine <robert@seamlessml.com >
Co-authored-by: root <rirv938@gmail.com >
Co-authored-by: Casper <casperbh.96@gmail.com >
Co-authored-by: julian-q <julianhquevedo@gmail.com >
2023-09-16 00:03:37 -07:00
ab019eea75
Add Model Revision Support ( #1014 )
...
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com >
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com >
2023-09-13 15:20:02 -07:00
Zhuohan Li
c957c741d9
Enable safetensors loading for all models ( #974 )
2023-09-07 15:49:52 -07:00
Antoni Baum
005ba458b5
Set torch default dtype in a context manager ( #971 )
...
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com >
2023-09-07 15:39:37 +09:00
shunxing1234 and ftgreat
ad5f2fe34c
Add support for aquila ( #663 )
...
* add aquila
Signed-off-by: ftgreat <ftgreat@163.com >
* fix some bug
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* delete pdb
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* delete whitespace
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* format
* fix order
---------
Signed-off-by: ftgreat <ftgreat@163.com >
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
Co-authored-by: ftgreat <ftgreat@163.com >
2023-08-22 00:13:36 -07:00
Jia Guoqing
735ecfff61
add internlm model ( #528 )
2023-08-08 16:35:06 -07:00
Qing and wq.chu
a57d13cc96
add QWen-7b ( #685 )
...
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com >
2023-08-08 13:50:38 -07:00
Zhuohan Li
1b0bd0fe8a
Add Falcon support (new) ( #592 )
2023-08-02 14:04:39 -07:00
Song
64f23c2900
fix baichuan for different position embedding for 7b and 13b models ( #643 )
2023-08-01 22:22:51 -07:00
codethazine
20b0d88d16
Add support for baichuan ( #365 )
2023-07-17 13:50:55 -07:00
Andre Slavescu and woWoosuk Kwon
c894836108
[Model] Add support for GPT-J ( #226 )
...
Co-authored-by: woWoosuk Kwon <woosuk.kwon@berkeley.edu >
2023-07-08 17:55:16 -07:00
Woosuk Kwon
404422f42e
[Model] Add support for MPT ( #334 )
2023-07-03 16:47:53 -07:00
Woosuk Kwon
e41f06702c
Add support for BLOOM ( #331 )
2023-07-03 13:12:35 -07:00
Zhuohan Li
d6fa1be3a8
[Quality] Add code formatter and linter ( #326 )
2023-07-03 11:31:55 -07:00
BasicCoder
471a7a4566
Compatible with Decapoda Research llama hf version ( #251 )
2023-06-26 09:23:57 -07:00
Michael Feil
298695b766
GPTBigCode (StarCoder, SantaCoder Support) ( #209 )
2023-06-23 01:49:27 +08:00
Woosuk Kwon
0b98ba15c7
Change the name to vLLM ( #150 )
2023-06-17 03:07:40 -07:00
