Optimize Mixtral with expert parallelism (#2090)

This commit is contained in:
Antoni Baum
2023-12-13 23:55:07 -08:00
committed by GitHub
parent f1c8520146
commit 21d93c140d
6 changed files with 230 additions and 343 deletions
-4
View File
@@ -72,10 +72,6 @@ Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/get
```bash
pip install vllm
```
**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks):
```bash
pip install megablocks
```
## Getting Started