 Gary Hui
|
7878958c0d
|
Address Phi modeling update 2 (#2428)
|
2024-01-12 12:16:49 -08:00 |
|
|
Woosuk Kwon
|
50376faa7b
|
Rename phi_1_5 -> phi (#2385)
|
2024-01-11 16:23:43 -08:00 |
|
|
Cade Daniel
|
79d64c4954
|
[Speculative decoding 1/9] Optimized rejection sampler (#2336)
|
2024-01-09 15:38:41 -08:00 |
|
|
Woosuk Kwon
|
28c3f12104
|
[Minor] Remove unused code in attention (#2384)
|
2024-01-08 13:13:08 -08:00 |
|
|
Zhuohan Li
|
fd4ea8ef5c
|
Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221)
|
2024-01-03 11:30:22 -08:00 |
|
|
Roy
|
9140561059
|
[Minor] Fix typo and remove unused code (#2305)
|
2024-01-02 19:23:15 -08:00 |
|
|
Jong-hun Shin
|
4934d49274
|
Support GPT-NeoX Models without attention biases (#2301)
|
2023-12-30 11:42:04 -05:00 |
|
|
Antoni Baum
|
bd29cf3d3a
|
Remove Sampler copy stream (#2209)
|
2023-12-20 00:04:33 -08:00 |
|
|
Woosuk Kwon
|
ba4f826738
|
[BugFix] Fix weight loading for Mixtral with TP (#2208)
|
2023-12-19 16:16:11 -08:00 |
|
|
avideci
|
de60a3fb93
|
Added DeciLM-7b and DeciLM-7b-instruct (#2062)
|
2023-12-19 02:29:33 -08:00 |
|
|
Woosuk Kwon
|
2c9b638065
|
[Minor] Fix a typo in .pt weight support (#2160)
|
2023-12-17 10:12:44 -08:00 |
|
|
Antoni Baum
|
a7347d9a6d
|
Make sampler less blocking (#1889)
|
2023-12-17 23:03:49 +08:00 |
|
|
Woosuk Kwon
|
c3372e87be
|
Remove dependency on CuPy (#2152)
|
2023-12-17 01:49:07 -08:00 |
|
|
|
37ca558103
|
Optimize model execution with CUDA graph (#1926)
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
|
2023-12-16 21:12:08 -08:00 |
|
|
Roy
|
eed74a558f
|
Simplify weight loading logic (#2133)
|
2023-12-16 12:41:23 -08:00 |
|
|
CHU Tianxiang
|
0fbfc4b81b
|
Add GPTQ support (#916)
|
2023-12-15 03:04:22 -08:00 |
|
|
Antoni Baum
|
21d93c140d
|
Optimize Mixtral with expert parallelism (#2090)
|
2023-12-13 23:55:07 -08:00 |
|
|
Woosuk Kwon
|
518369d78c
|
Implement lazy model loader (#2044)
|
2023-12-12 22:21:45 -08:00 |
|
|
Megha AgarwalandWoosuk Kwon
|
6428f1d051
|
Support MPT with GQA (#1938)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2023-12-12 10:16:05 -08:00 |
|
|
Woosuk Kwon
|
cb3f30c600
|
Upgrade transformers version to 4.36.0 (#2046)
|
2023-12-11 18:39:14 -08:00 |
|
|
Woosuk Kwon
|
31d2ab4aff
|
Remove python 3.10 requirement (#2040)
|
2023-12-11 12:26:42 -08:00 |
|
|
Woosuk Kwon
|
6120e5aaea
|
Fix import error msg for megablocks (#2038)
|
2023-12-11 11:40:56 -08:00 |
|
|
Woosuk Kwon
|
81ce2a4b26
|
[Minor] Fix type annotation in Mixtral (#2036)
|
2023-12-11 11:32:39 -08:00 |
|
|
Woosuk Kwon
|
4ff0203987
|
Minor fixes for Mixtral (#2015)
|
2023-12-11 09:16:15 -08:00 |
|
|
|
b5f882cc98
|
Mixtral 8x7B support (#2011)
Co-authored-by: Pierre Stock <p@mistral.ai>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
|
2023-12-11 01:09:15 -08:00 |
|
|
|
dacaf5a400
|
Replace head_mapping params with num_kv_heads to attention kernel. (#1997)
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
|
2023-12-10 10:12:53 -08:00 |
|
|
Woosuk Kwon
|
24cde76a15
|
[Minor] Add comment on skipping rope caches (#2004)
|
2023-12-10 10:04:12 -08:00 |
|
|
Woosuk Kwon
|
fe470ae5ad
|
[Minor] Fix code style for baichuan (#2003)
|
2023-12-09 19:24:29 -08:00 |
|
|
Jun Gao
|
3a8c2381f7
|
Fix for KeyError on Loading LLaMA (#1978)
|
2023-12-09 15:59:57 -08:00 |
|
|
firebook
|
2b981012a6
|
Fix Baichuan2-7B-Chat (#1987)
|
2023-12-08 09:38:36 -08:00 |
|
|
|
6ccc0bfffb
|
Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836)
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: root <kuanfu.liu@akirakan.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com>
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com>
|
2023-12-07 23:16:52 -08:00 |
|
|
Jie Liandlijie8
|
ebede26ebf
|
Make InternLM follow rope_scaling in config.json (#1956)
Co-authored-by: lijie8 <lijie8@sensetime.com>
|
2023-12-07 08:32:08 -08:00 |
|
|
dancingpipiandshujunhua1
|
1d9b737e05
|
Support ChatGLMForConditionalGeneration (#1932)
Co-authored-by: shujunhua1 <shujunhua1@jd.com>
|
2023-12-05 10:52:48 -08:00 |
|
|
Woosuk Kwon
|
9b294976a2
|
Add PyTorch-native implementation of custom layers (#1898)
|
2023-12-02 21:18:40 -08:00 |
|
|
Woosuk Kwon
|
e5452ddfd6
|
Normalize head weights for Baichuan 2 (#1876)
|
2023-11-30 20:03:58 -08:00 |
|
|
Roy
|
d27f4bae39
|
Fix rope cache key error (#1867)
|
2023-11-30 08:29:28 -08:00 |
|
|
Jee Li
|
63b2206ad0
|
Avoid multiple instantiations of the RoPE class (#1828)
|
2023-11-29 23:06:27 -08:00 |
|
|
Woosuk Kwon
|
27feead2f8
|
Refactor Worker & InputMetadata (#1843)
|
2023-11-29 22:16:37 -08:00 |
|
|
Woosuk Kwon
|
a9e4574261
|
Refactor Attention (#1840)
|
2023-11-29 15:37:31 -08:00 |
|
|
Woosuk Kwon
|
a7b3e33078
|
[Fix] Fix RoPE in ChatGLM-32K (#1841)
|
2023-11-29 13:01:19 -08:00 |
|
|
Zhuohan Li
|
708e6c18b0
|
[FIX] Fix class naming (#1803)
|
2023-11-28 14:08:01 -08:00 |
|
|
Woosuk Kwon
|
b943890484
|
Fix OPT param names (#1819)
|
2023-11-28 11:22:44 -08:00 |
|
|
explainerauthors
|
a1125ad4df
|
Correct comments in parallel_state.py (#1818)
|
2023-11-28 10:19:35 -08:00 |
|
|
ljss
|
a8b150c595
|
Init model on GPU to reduce CPU memory footprint (#1796)
|
2023-11-27 11:18:26 -08:00 |
|
|
Woosuk Kwon
|
7c600440f7
|
Fix model docstrings (#1764)
|
2023-11-23 23:04:44 -08:00 |
|
|
Yanming W
|
e0c6f556e8
|
[Build] Avoid building too many extensions (#1624)
|
2023-11-23 16:31:19 -08:00 |
|
|
ljss
|
de23687d16
|
Fix repetition penalty aligned with huggingface (#1577)
|
2023-11-22 14:41:44 -08:00 |
|
|
Zhuohan Li
|
7d761fe3c1
|
[FIX] Fix the case when input_is_parallel=False for ScaledActivation (#1737)
|
2023-11-20 23:56:48 -08:00 |
|
|
Woosuk Kwon
|
cf35d8f3d7
|
[BugFix] Fix TP support for AWQ (#1731)
|
2023-11-20 21:42:45 -08:00 |
|
|
ljss
|
819b18e7ba
|
Rewrite torch.repeat_interleave to remove cpu synchronization (#1599)
|
2023-11-20 17:46:32 -08:00 |
|