ljss
e1054247ba
[Optimization] Implement fused add rmsnorm ( #1667 )
2023-11-18 18:18:02 -08:00
Woosuk Kwon
8d17774f92
Add AWQ support for all models ( #1714 )
2023-11-18 17:56:47 -08:00
maximzubkov
521b35f799
Support Microsoft Phi 1.5 ( #1664 )
2023-11-16 14:28:39 -08:00
Megha Agarwal
b514d3c496
Revert MptConfig to MPTConfig ( #1668 )
2023-11-16 01:19:39 -08:00
Zhuohan Li
7076fa1c9f
TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models ( #1622 )
...
Refactor the tensor parallelism, quantization, and weight-loading codes.
Summary of the new features enabled by this PR:
- **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580 ).
- Model loading code became much simpler.
- Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.
2023-11-15 22:50:41 -08:00
lirui
eb825c1e74
Fix #1474 - AssertionError:assert param_slice.shape == loaded_weight.shape ( #1631 )
2023-11-12 15:53:12 -08:00
GoHomeToMacDonal
1a2bbc9301
ChatGLM Support ( #1261 )
2023-11-06 16:09:33 -08:00
Roy
e7f579eb97
Support Yi model ( #1567 )
2023-11-06 15:26:03 -08:00
Woosuk Kwon
1fe0990023
Remove MPTConfig ( #1529 )
2023-11-01 15:29:05 -07:00
Woosuk Kwon
aa9af07cac
Fix bias in InternLM ( #1501 )
2023-10-29 16:24:18 -07:00
Qing
28b47d1e49
Add rope_scaling to Aquila model ( #1457 )
2023-10-29 04:25:21 -07:00
1f24755bf8
Support SqueezeLLM ( #1326 )
...
Co-authored-by: squeeze-ai-lab <squeezeailab.bair@gmail.com >
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2023-10-21 23:14:59 -07:00
Lu Wang
de89472897
Fix the issue for AquilaChat2-* models ( #1339 )
2023-10-13 11:51:29 -07:00
Woosuk Kwon
e7c8555d06
Bump up transformers version & Remove MistralConfig ( #1254 )
2023-10-13 10:05:26 -07:00
Zhuohan Li
b95ee898fe
[Minor] Fix comment in mistral.py ( #1303 )
2023-10-09 19:44:37 -07:00
Zhuohan Li
ba0bfd40e2
TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic ( #1181 )
2023-10-02 15:36:09 -07:00
Zhuohan Li and JiCheng
a60b353005
support sharding llama2-70b on more than 8 GPUs ( #1209 )
...
Co-authored-by: JiCheng <247153481@qq.com >
2023-10-02 15:26:33 -07:00
Woosuk Kwon
a8e98aee0c
Fix Mistral model ( #1220 )
2023-09-28 10:44:05 -07:00
Chris Bamford and timlacroix
bb1ba58f06
[Mistral] Mistral-7B-v0.1 support ( #1196 )
...
Co-authored-by: timlacroix <t@mistral.ai >
2023-09-28 10:41:03 -07:00
Qing
7bedab5748
Add rope_scaling to Qwen ( #1210 )
2023-09-28 00:49:23 -07:00
Qing
28e616c4e3
fix qwen-14b model ( #1173 )
2023-09-27 16:33:16 -07:00
21877b0d75
Support Longchat and RoPE scaling ( #555 )
...
Co-authored-by: Wing Lian <wing.lian@gmail.com >
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2023-09-27 03:36:02 -07:00
3302f0aef3
rope_theta and max_position_embeddings from config ( #1096 )
...
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
Co-authored-by: wnma3mz <wnma3mz@gmail.com >
2023-09-20 13:35:11 -07:00
Woosuk Kwon
cc796b1358
Convert before transpose ( #1073 )
2023-09-18 11:51:48 -07:00
e3e79e9e8a
Implement AWQ quantization support for LLaMA ( #1032 )
...
Co-authored-by: Robert Irvine <robert@seamlessml.com >
Co-authored-by: root <rirv938@gmail.com >
Co-authored-by: Casper <casperbh.96@gmail.com >
Co-authored-by: julian-q <julianhquevedo@gmail.com >
2023-09-16 00:03:37 -07:00
ab019eea75
Add Model Revision Support ( #1014 )
...
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com >
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com >
2023-09-13 15:20:02 -07:00
Zhuohan Li
c957c741d9
Enable safetensors loading for all models ( #974 )
2023-09-07 15:49:52 -07:00
Woosuk Kwon
320a622ec4
[BugFix] Implement RoPE for GPT-J ( #941 )
2023-09-06 11:54:33 +09:00
Zhuohan Li
002800f081
Align vLLM's beam search implementation with HF generate ( #857 )
2023-09-04 17:29:42 -07:00
JFDuan
0d93f15694
Accelerate LLaMA model loading ( #234 )
2023-08-30 01:00:13 -07:00
Antoni Baum
4b6f069b6f
Add support for CodeLlama ( #854 )
2023-08-25 12:44:07 -07:00
Wen Sun
eedac9dba0
fix: revert code to avoid no attribute problem ( #827 )
2023-08-22 11:55:16 -07:00
shunxing1234 and ftgreat
ad5f2fe34c
Add support for aquila ( #663 )
...
* add aquila
Signed-off-by: ftgreat <ftgreat@163.com >
* fix some bug
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* delete pdb
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* delete whitespace
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
* format
* fix order
---------
Signed-off-by: ftgreat <ftgreat@163.com >
Signed-off-by: shunxing1234 <xw747777271@gmail.com >
Co-authored-by: ftgreat <ftgreat@163.com >
2023-08-22 00:13:36 -07:00
zhaoyang-star
4f8584756d
Fix mqa is false case in gpt_bigcode ( #806 )
2023-08-21 22:22:06 -07:00
WRH
462ae5220a
[Fix] unwantted bias in InternLM Model ( #740 )
2023-08-11 11:40:37 -07:00
Jia Guoqing
735ecfff61
add internlm model ( #528 )
2023-08-08 16:35:06 -07:00
Qing and wq.chu
a57d13cc96
add QWen-7b ( #685 )
...
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com >
2023-08-08 13:50:38 -07:00
Zhuohan Li
f7389f4763
[Doc] Add Baichuan 13B to supported models ( #656 )
2023-08-02 16:45:12 -07:00
Zhuohan Li
1b0bd0fe8a
Add Falcon support (new) ( #592 )
2023-08-02 14:04:39 -07:00
Song
64f23c2900
fix baichuan for different position embedding for 7b and 13b models ( #643 )
2023-08-01 22:22:51 -07:00
Qing and wq.chu
d4c7755ca8
fix biachuan-7b tp ( #598 )
...
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com >
2023-08-01 15:41:36 -07:00
Zhuohan Li
7d5a155e4a
[Fix] Fix GPTBigcoder for distributed execution ( #503 )
2023-07-24 18:36:33 -07:00
leegohi04517
1dde34e0f8
GPTJConfig has no attribute rotary. ( #532 )
2023-07-24 11:29:30 -07:00
Zhuohan Li
6fc2a38b11
Add support for LLaMA-2 ( #505 )
2023-07-20 11:38:27 -07:00
codethazine
20b0d88d16
Add support for baichuan ( #365 )
2023-07-17 13:50:55 -07:00
Zhuohan Li
96853af5a8
Optimize MQA Kernel ( #452 )
2023-07-14 20:06:40 -04:00
Wen Sun
dbed69058c
Fix the KeyError when loading bloom-based models ( #441 )
2023-07-13 21:58:09 -07:00
panda
7b6ae94059
add vocab padding for LLama(Support WizardLM) ( #411 )
2023-07-13 23:56:22 -04:00
Andre Slavescu and woWoosuk Kwon
c894836108
[Model] Add support for GPT-J ( #226 )
...
Co-authored-by: woWoosuk Kwon <woosuk.kwon@berkeley.edu >
2023-07-08 17:55:16 -07:00
Woosuk Kwon
404422f42e
[Model] Add support for MPT ( #334 )
2023-07-03 16:47:53 -07:00
ljss