Pernekhan Utemuratov and Pernekhan Utemuratov
c410f5d020
Use revision when downloading the quantization config file ( #2697 )
...
Co-authored-by: Pernekhan Utemuratov <pernekhan@deepinfra.com >
2024-02-01 15:41:58 -08:00
Roy
91a61da9b1
[Bugfix] fix load local safetensors model ( #2512 )
2024-01-19 16:26:16 -08:00
Simon Mo
dd7e8f5f64
refactor complemention api for readability ( #2499 )
2024-01-18 16:45:14 -08:00
Nikola Borisov
7e1081139d
Don't download both safetensor and bin files. ( #2480 )
2024-01-18 11:05:53 -08:00
Woosuk Kwon
2c9b638065
[Minor] Fix a typo in .pt weight support ( #2160 )
2023-12-17 10:12:44 -08:00
Roy
eed74a558f
Simplify weight loading logic ( #2133 )
2023-12-16 12:41:23 -08:00
CHU Tianxiang
0fbfc4b81b
Add GPTQ support ( #916 )
2023-12-15 03:04:22 -08:00
Woosuk Kwon
f5a37c6c6c
[BugFix] Fix a bug in loading safetensors ( #1732 )
2023-11-20 15:51:18 -08:00
Simon Mo
5ffc0d13a2
Migrate linter from pylint to ruff ( #1665 )
2023-11-20 11:58:01 -08:00
twaka
e946260cf3
use get_tensor in safe_open ( #1696 )
2023-11-18 16:45:18 -08:00
Woosuk Kwon
bb00f66e19
Use quantization_config in hf config ( #1695 )
2023-11-17 16:23:49 -08:00
twaka
2a2c135b41
Fix loading error when safetensors contains empty tensor ( #1687 )
2023-11-16 10:38:10 -08:00
Zhuohan Li
7076fa1c9f
TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models ( #1622 )
...
Refactor the tensor parallelism, quantization, and weight-loading codes.
Summary of the new features enabled by this PR:
- **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580 ).
- Model loading code became much simpler.
- Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.
2023-11-15 22:50:41 -08:00
Woosuk Kwon
875afe38ab
Add blacklist in model checkpoint ( #1325 )
2023-10-12 01:05:37 -07:00
e3e79e9e8a
Implement AWQ quantization support for LLaMA ( #1032 )
...
Co-authored-by: Robert Irvine <robert@seamlessml.com >
Co-authored-by: root <rirv938@gmail.com >
Co-authored-by: Casper <casperbh.96@gmail.com >
Co-authored-by: julian-q <julianhquevedo@gmail.com >
2023-09-16 00:03:37 -07:00
ab019eea75
Add Model Revision Support ( #1014 )
...
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com >
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com >
2023-09-13 15:20:02 -07:00
Zhuohan Li
c957c741d9
Enable safetensors loading for all models ( #974 )
2023-09-07 15:49:52 -07:00
JFDuan
0d93f15694
Accelerate LLaMA model loading ( #234 )
2023-08-30 01:00:13 -07:00
Xinyu Yang
73b3de79ea
explicitly del state ( #784 )
2023-08-17 12:56:04 -07:00
Fazlul Shahriar
75beba29b5
Don't try to load training_args.bin ( #373 )
2023-07-08 15:26:28 -07:00
Zhuohan Li
d6fa1be3a8
[Quality] Add code formatter and linter ( #326 )
2023-07-03 11:31:55 -07:00
Zhuohan Li
598dc4b79a
[Fix] Weight loading for GPTBigCode ( #313 )
2023-06-29 22:14:17 -07:00
Woosuk Kwon
0b98ba15c7
Change the name to vLLM ( #150 )
2023-06-17 03:07:40 -07:00
