Optimize Mixtral with expert parallelism (#2090)
This commit is contained in:
@@ -72,10 +72,6 @@ Install vLLM with pip or [from source](https://vllm.readthedocs.io/en/latest/get
|
||||
```bash
|
||||
pip install vllm
|
||||
```
|
||||
**NOTE:** The Mixtral model additionally requires `megablocks` which can be installed with pip or [from source](https://github.com/stanford-futuredata/megablocks):
|
||||
```bash
|
||||
pip install megablocks
|
||||
```
|
||||
|
||||
## Getting Started
|
||||
|
||||
|
||||
Reference in New Issue
Block a user