YingchaoX
8a25d3a71a
fix stablelm.py tensor-parallel-size bug ( #2482 )
2024-01-18 09:39:46 -08:00
Hyunsung Lee
e1957c6ebd
Add StableLM3B model ( #2372 )
2024-01-16 20:32:40 -08:00
Gary Hui
7878958c0d
Address Phi modeling update 2 ( #2428 )
2024-01-12 12:16:49 -08:00
Woosuk Kwon
50376faa7b
Rename phi_1_5 -> phi ( #2385 )
2024-01-11 16:23:43 -08:00
Zhuohan Li
fd4ea8ef5c
Use NCCL instead of ray for control-plane communication to remove serialization overhead ( #2221 )
2024-01-03 11:30:22 -08:00
Jong-hun Shin
4934d49274
Support GPT-NeoX Models without attention biases ( #2301 )
2023-12-30 11:42:04 -05:00
Woosuk Kwon
ba4f826738
[BugFix] Fix weight loading for Mixtral with TP ( #2208 )
2023-12-19 16:16:11 -08:00
avideci
de60a3fb93
Added DeciLM-7b and DeciLM-7b-instruct ( #2062 )
2023-12-19 02:29:33 -08:00
37ca558103
Optimize model execution with CUDA graph ( #1926 )
...
Co-authored-by: Chen Shen <scv119@gmail.com >
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com >
2023-12-16 21:12:08 -08:00
Roy
eed74a558f
Simplify weight loading logic ( #2133 )
2023-12-16 12:41:23 -08:00
CHU Tianxiang
0fbfc4b81b
Add GPTQ support ( #916 )
2023-12-15 03:04:22 -08:00
Antoni Baum
21d93c140d
Optimize Mixtral with expert parallelism ( #2090 )
2023-12-13 23:55:07 -08:00
Woosuk Kwon
518369d78c
Implement lazy model loader ( #2044 )
2023-12-12 22:21:45 -08:00
Megha Agarwal and Woosuk Kwon
6428f1d051
Support MPT with GQA ( #1938 )
...
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2023-12-12 10:16:05 -08:00
Woosuk Kwon
cb3f30c600
Upgrade transformers version to 4.36.0 ( #2046 )
2023-12-11 18:39:14 -08:00
Woosuk Kwon
31d2ab4aff
Remove python 3.10 requirement ( #2040 )
2023-12-11 12:26:42 -08:00
Woosuk Kwon
6120e5aaea
Fix import error msg for megablocks ( #2038 )
2023-12-11 11:40:56 -08:00
Woosuk Kwon
81ce2a4b26
[Minor] Fix type annotation in Mixtral ( #2036 )
2023-12-11 11:32:39 -08:00
Woosuk Kwon
4ff0203987
Minor fixes for Mixtral ( #2015 )
2023-12-11 09:16:15 -08:00
b5f882cc98
Mixtral 8x7B support ( #2011 )
...
Co-authored-by: Pierre Stock <p@mistral.ai >
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com >
2023-12-11 01:09:15 -08:00
Woosuk Kwon
24cde76a15
[Minor] Add comment on skipping rope caches ( #2004 )
2023-12-10 10:04:12 -08:00
Woosuk Kwon
fe470ae5ad
[Minor] Fix code style for baichuan ( #2003 )
2023-12-09 19:24:29 -08:00
Jun Gao
3a8c2381f7
Fix for KeyError on Loading LLaMA ( #1978 )
2023-12-09 15:59:57 -08:00
firebook
2b981012a6
Fix Baichuan2-7B-Chat ( #1987 )
2023-12-08 09:38:36 -08:00
Jie Li and lijie8
ebede26ebf
Make InternLM follow rope_scaling in config.json ( #1956 )
...
Co-authored-by: lijie8 <lijie8@sensetime.com >
2023-12-07 08:32:08 -08:00
Woosuk Kwon
e5452ddfd6
Normalize head weights for Baichuan 2 ( #1876 )
2023-11-30 20:03:58 -08:00
Woosuk Kwon
27feead2f8
Refactor Worker & InputMetadata ( #1843 )
2023-11-29 22:16:37 -08:00
Woosuk Kwon
a9e4574261
Refactor Attention ( #1840 )
2023-11-29 15:37:31 -08:00
Woosuk Kwon
a7b3e33078
[Fix] Fix RoPE in ChatGLM-32K ( #1841 )
2023-11-29 13:01:19 -08:00
Woosuk Kwon
b943890484
Fix OPT param names ( #1819 )
2023-11-28 11:22:44 -08:00
Woosuk Kwon
7c600440f7
Fix model docstrings ( #1764 )
2023-11-23 23:04:44 -08:00
Woosuk Kwon
cf35d8f3d7
[BugFix] Fix TP support for AWQ ( #1731 )
2023-11-20 21:42:45 -08:00
Simon Mo
5ffc0d13a2
Migrate linter from pylint to ruff ( #1665 )
2023-11-20 11:58:01 -08:00
ljss
e1054247ba
[Optimization] Implement fused add rmsnorm ( #1667 )
2023-11-18 18:18:02 -08:00
Woosuk Kwon
8d17774f92
Add AWQ support for all models ( #1714 )
2023-11-18 17:56:47 -08:00
maximzubkov
521b35f799
Support Microsoft Phi 1.5 ( #1664 )
2023-11-16 14:28:39 -08:00
Megha Agarwal
b514d3c496
Revert MptConfig to MPTConfig ( #1668 )
2023-11-16 01:19:39 -08:00
Zhuohan Li
7076fa1c9f
TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models ( #1622 )
...
Refactor the tensor parallelism, quantization, and weight-loading codes.
Summary of the new features enabled by this PR:
- **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580 ).
- Model loading code became much simpler.
- Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.
2023-11-15 22:50:41 -08:00
lirui
eb825c1e74
Fix #1474 - AssertionError:assert param_slice.shape == loaded_weight.shape ( #1631 )
2023-11-12 15:53:12 -08:00
GoHomeToMacDonal
1a2bbc9301
ChatGLM Support ( #1261 )
2023-11-06 16:09:33 -08:00
Roy
e7f579eb97
Support Yi model ( #1567 )
2023-11-06 15:26:03 -08:00
Woosuk Kwon
1fe0990023
Remove MPTConfig ( #1529 )
2023-11-01 15:29:05 -07:00
Woosuk Kwon
aa9af07cac
Fix bias in InternLM ( #1501 )
2023-10-29 16:24:18 -07:00
Qing
28b47d1e49
Add rope_scaling to Aquila model ( #1457 )
2023-10-29 04:25:21 -07:00
1f24755bf8
Support SqueezeLLM ( #1326 )
...
Co-authored-by: squeeze-ai-lab <squeezeailab.bair@gmail.com >
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2023-10-21 23:14:59 -07:00
Lu Wang
de89472897
Fix the issue for AquilaChat2-* models ( #1339 )
2023-10-13 11:51:29 -07:00
Woosuk Kwon
e7c8555d06
Bump up transformers version & Remove MistralConfig ( #1254 )
2023-10-13 10:05:26 -07:00
Zhuohan Li
b95ee898fe
[Minor] Fix comment in mistral.py ( #1303 )
2023-10-09 19:44:37 -07:00
Zhuohan Li
ba0bfd40e2
TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic ( #1181 )
2023-10-02 15:36:09 -07:00
Zhuohan Li and JiCheng
a60b353005
support sharding llama2-70b on more than 8 GPUs ( #1209 )
...
Co-authored-by: JiCheng <247153481@qq.com >
2023-10-02 15:26:33 -07:00
YingchaoX