Woosuk Kwon
cb3f30c600
Upgrade transformers version to 4.36.0 ( #2046 )
2023-12-11 18:39:14 -08:00
Woosuk Kwon
31d2ab4aff
Remove python 3.10 requirement ( #2040 )
2023-12-11 12:26:42 -08:00
Woosuk Kwon
6120e5aaea
Fix import error msg for megablocks ( #2038 )
2023-12-11 11:40:56 -08:00
Woosuk Kwon
81ce2a4b26
[Minor] Fix type annotation in Mixtral ( #2036 )
2023-12-11 11:32:39 -08:00
Woosuk Kwon
4ff0203987
Minor fixes for Mixtral ( #2015 )
2023-12-11 09:16:15 -08:00
b5f882cc98
Mixtral 8x7B support ( #2011 )
...
Co-authored-by: Pierre Stock <p@mistral.ai >
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com >
2023-12-11 01:09:15 -08:00
dacaf5a400
Replace head_mapping params with num_kv_heads to attention kernel. ( #1997 )
...
Co-authored-by: wangguoya <wangguoya@baidu.com >
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com >
2023-12-10 10:12:53 -08:00
Woosuk Kwon
24cde76a15
[Minor] Add comment on skipping rope caches ( #2004 )
2023-12-10 10:04:12 -08:00
Woosuk Kwon
fe470ae5ad
[Minor] Fix code style for baichuan ( #2003 )
2023-12-09 19:24:29 -08:00
Jun Gao
3a8c2381f7
Fix for KeyError on Loading LLaMA ( #1978 )
2023-12-09 15:59:57 -08:00
firebook
2b981012a6
Fix Baichuan2-7B-Chat ( #1987 )
2023-12-08 09:38:36 -08:00
6ccc0bfffb
Merge EmbeddedLLM/vllm-rocm into vLLM main ( #1836 )
...
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com >
Co-authored-by: Amir Balwel <amoooori04@gmail.com >
Co-authored-by: root <kuanfu.liu@akirakan.com >
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com >
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com >
2023-12-07 23:16:52 -08:00
Jie Li and lijie8
ebede26ebf
Make InternLM follow rope_scaling in config.json ( #1956 )
...
Co-authored-by: lijie8 <lijie8@sensetime.com >
2023-12-07 08:32:08 -08:00
dancingpipi and shujunhua1
1d9b737e05
Support ChatGLMForConditionalGeneration ( #1932 )
...
Co-authored-by: shujunhua1 <shujunhua1@jd.com >
2023-12-05 10:52:48 -08:00
Woosuk Kwon
9b294976a2
Add PyTorch-native implementation of custom layers ( #1898 )
2023-12-02 21:18:40 -08:00
Woosuk Kwon
e5452ddfd6
Normalize head weights for Baichuan 2 ( #1876 )
2023-11-30 20:03:58 -08:00
Roy
d27f4bae39
Fix rope cache key error ( #1867 )
2023-11-30 08:29:28 -08:00
Jee Li
63b2206ad0
Avoid multiple instantiations of the RoPE class ( #1828 )
2023-11-29 23:06:27 -08:00
Woosuk Kwon
27feead2f8
Refactor Worker & InputMetadata ( #1843 )
2023-11-29 22:16:37 -08:00
Woosuk Kwon
a9e4574261
Refactor Attention ( #1840 )
2023-11-29 15:37:31 -08:00
Woosuk Kwon
a7b3e33078
[Fix] Fix RoPE in ChatGLM-32K ( #1841 )
2023-11-29 13:01:19 -08:00
Zhuohan Li
708e6c18b0
[FIX] Fix class naming ( #1803 )
2023-11-28 14:08:01 -08:00
Woosuk Kwon
b943890484
Fix OPT param names ( #1819 )
2023-11-28 11:22:44 -08:00
explainerauthors
a1125ad4df
Correct comments in parallel_state.py ( #1818 )
2023-11-28 10:19:35 -08:00
ljss
a8b150c595
Init model on GPU to reduce CPU memory footprint ( #1796 )
2023-11-27 11:18:26 -08:00
Woosuk Kwon
7c600440f7
Fix model docstrings ( #1764 )
2023-11-23 23:04:44 -08:00
Yanming W
e0c6f556e8
[Build] Avoid building too many extensions ( #1624 )
2023-11-23 16:31:19 -08:00
ljss
de23687d16
Fix repetition penalty aligned with huggingface ( #1577 )
2023-11-22 14:41:44 -08:00
Zhuohan Li
7d761fe3c1
[FIX] Fix the case when input_is_parallel=False for ScaledActivation ( #1737 )
2023-11-20 23:56:48 -08:00
Woosuk Kwon
cf35d8f3d7
[BugFix] Fix TP support for AWQ ( #1731 )
2023-11-20 21:42:45 -08:00
ljss
819b18e7ba
Rewrite torch.repeat_interleave to remove cpu synchronization ( #1599 )
2023-11-20 17:46:32 -08:00
Woosuk Kwon
f5a37c6c6c
[BugFix] Fix a bug in loading safetensors ( #1732 )
2023-11-20 15:51:18 -08:00
Simon Mo
5ffc0d13a2
Migrate linter from pylint to ruff ( #1665 )
2023-11-20 11:58:01 -08:00
ljss
e1054247ba
[Optimization] Implement fused add rmsnorm ( #1667 )
2023-11-18 18:18:02 -08:00
Woosuk Kwon
8d17774f92
Add AWQ support for all models ( #1714 )
2023-11-18 17:56:47 -08:00
twaka
e946260cf3
use get_tensor in safe_open ( #1696 )
2023-11-18 16:45:18 -08:00
Woosuk Kwon
bb00f66e19
Use quantization_config in hf config ( #1695 )
2023-11-17 16:23:49 -08:00
Roy
e87557b069
Support Min P Sampler ( #1642 )
2023-11-17 16:20:49 -08:00
maximzubkov
521b35f799
Support Microsoft Phi 1.5 ( #1664 )
2023-11-16 14:28:39 -08:00
twaka
2a2c135b41
Fix loading error when safetensors contains empty tensor ( #1687 )
2023-11-16 10:38:10 -08:00
Megha Agarwal
b514d3c496
Revert MptConfig to MPTConfig ( #1668 )
2023-11-16 01:19:39 -08:00
Zhuohan Li
7076fa1c9f
TP/quantization/weight loading refactor part 2 - Refactor quantized linear logic and extend quantization support to all models ( #1622 )
...
Refactor the tensor parallelism, quantization, and weight-loading codes.
Summary of the new features enabled by this PR:
- **All models** are able to be quantized with AWQ and SqueezeLLM, and [soon GPTQ](https://github.com/vllm-project/vllm/pull/1580 ).
- Model loading code became much simpler.
- Support model parallelism for all MQA/GQA models when the number of key/value heads is smaller than the tensor parallel size.
2023-11-15 22:50:41 -08:00
Woosuk Kwon
054072bee5
[Minor] Move RoPE selection logic to get_rope ( #1633 )
2023-11-12 16:04:50 -08:00
lirui
eb825c1e74
Fix #1474 - AssertionError:assert param_slice.shape == loaded_weight.shape ( #1631 )
2023-11-12 15:53:12 -08:00
forpanyang
ab9e8488d5
Add Yi model to quantization support ( #1600 )
2023-11-09 11:47:14 -08:00
GoHomeToMacDonal
1a2bbc9301
ChatGLM Support ( #1261 )
2023-11-06 16:09:33 -08:00
Roy
e7f579eb97
Support Yi model ( #1567 )
2023-11-06 15:26:03 -08:00
9f669a9a7c
Support YaRN models ( #1264 )
...
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com >
Co-authored-by: Viktor Ferenczi <viktor@ferenczi.eu >
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
2023-11-03 14:12:48 -07:00
Noam Gat
555bdcc5a3
Added logits processor API to sampling params ( #1469 )
2023-11-03 14:12:15 -07:00
Antoni Baum
9738b84a08
Force paged attention v2 for long contexts ( #1510 )
2023-11-01 16:24:32 -07:00
Woosuk Kwon