Support starcoder2 architecture (#3089)

This commit is contained in:
Seonghyeon
2024-02-29 00:51:48 -08:00
committed by GitHub
parent 9289e577ec
commit bfdcfa6a05
7 changed files with 452 additions and 0 deletions
+1
View File
@@ -78,6 +78,7 @@ vLLM seamlessly supports many Hugging Face models, including the following archi
- Qwen (`Qwen/Qwen-7B`, `Qwen/Qwen-7B-Chat`, etc.)
- Qwen2 (`Qwen/Qwen2-7B-beta`, `Qwen/Qwen-7B-Chat-beta`, etc.)
- StableLM(`stabilityai/stablelm-3b-4e1t`, `stabilityai/stablelm-base-alpha-7b-v2`, etc.)
- Starcoder2(`bigcode/starcoder2-3b`, `bigcode/starcoder2-7b`, `bigcode/starcoder2-15b`, etc.)
- Yi (`01-ai/Yi-6B`, `01-ai/Yi-34B`, etc.)
Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/getting_started/installation.html#build-from-source):