[Docs] Add Apple Silicon documentation for vLLM-Metal GPU support (#41987)

Signed-off-by: alexagriffith <agriffith96@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Alexa Griffith
2026-05-11 11:34:25 -07:00
committed by GitHub
co-authored by Claude Sonnet 4.5
parent 5497ffbf7c
commit cf0d279142
4 changed files with 176 additions and 3 deletions
+4 -3
View File
@@ -3,9 +3,10 @@
vLLM supports the following hardware platforms:
- [GPU](gpu.md)
- [NVIDIA CUDA](gpu.md#nvidia-cuda)
- [AMD ROCm](gpu.md#amd-rocm)
- [Intel XPU](gpu.md#intel-xpu)
- [NVIDIA CUDA](gpu.md)
- [AMD ROCm](gpu.md)
- [Intel XPU](gpu.md)
- [Apple Silicon](gpu.md) (via [vLLM-Metal](https://github.com/vllm-project/vllm-metal))
- [CPU](cpu.md)
- [Intel/AMD x86](cpu.md#intelamd-x86)
- [ARM AArch64](cpu.md#arm-aarch64)
@@ -0,0 +1,125 @@
<!-- markdownlint-disable MD041 -->
--8<-- [start:installation]
For GPU-accelerated inference on Apple Silicon, use [vLLM-Metal](https://github.com/vllm-project/vllm-metal), a community-maintained hardware plugin that uses MLX as the compute backend and provides native GPU acceleration via Apple's Metal framework.
vLLM-Metal works with MLX-optimized models from the [mlx-community](https://huggingface.co/mlx-community) organization on Hugging Face, which provides quantized versions of popular models optimized for Apple Silicon.
!!! tip
For installation and usage instructions, see the [Set up using vLLM-Metal](#set-up-using-vllm-metal) section below.
--8<-- [end:installation]
--8<-- [start:requirements]
- OS: macOS Sonoma or later
- Hardware: Apple Silicon
- Metal support enabled
!!! note
See the [Set up using vLLM-Metal](#set-up-using-vllm-metal) section below for installation instructions.
--8<-- [end:requirements]
--8<-- [start:set-up-using-python]
## Set up using vLLM-Metal
vLLM-Metal is distributed as a separate package that provides native GPU acceleration on Apple Silicon.
To install vLLM-Metal, follow the installation instructions in the [vLLM-Metal documentation](https://github.com/vllm-project/vllm-metal#installation).
The installation will:
1. Set up the appropriate Python environment
2. Install MLX and required dependencies
3. Install the vLLM-Metal package
After installation, you can start using vLLM with Metal GPU acceleration.
!!! tip
When using vLLM-Metal, use models from the [mlx-community](https://huggingface.co/mlx-community) on Hugging Face for best performance. These models are optimized for MLX and often include quantized versions (4-bit, 8-bit) that run efficiently on Apple Silicon.
Example model: `mlx-community/Qwen2.5-0.5B-Instruct-4bit`
### Using vLLM-Metal
After installation, vLLM-Metal provides an easy-to-use CLI for running an OpenAI-compatible API server:
```bash
# Activate the vLLM-Metal environment
source ~/.venv-vllm-metal/bin/activate
# Start the API server (specify your mlx-community model or it will use default)
vllm serve
```
Once the server is running, you have multiple options to interact with it:
#### Option 1: Interactive chat
Open a new terminal and start an interactive chat session:
```bash
source ~/.venv-vllm-metal/bin/activate
vllm chat
```
#### Option 2: API requests with curl
```bash
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [{"role": "user", "content": "Hello!"}],
"max_tokens": 50
}'
```
#### Option 3: Python with OpenAI SDK
```python
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="dummy" # No auth required for local server
)
response = client.chat.completions.create(
model="mlx-community/Qwen2.5-0.5B-Instruct-4bit",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
```
For more details on the `vllm` CLI commands, see the [OpenAI-compatible server documentation](../../serving/openai_compatible_server.md).
--8<-- [end:set-up-using-python]
--8<-- [start:pre-built-wheels]
vLLM-Metal is installed via the vLLM-Metal package. See the [Set up using vLLM-Metal](#set-up-using-vllm-metal) section above.
--8<-- [end:pre-built-wheels]
--8<-- [start:build-wheel-from-source]
For build instructions from source, refer to the [vLLM-Metal documentation](https://github.com/vllm-project/vllm-metal#installation).
--8<-- [end:build-wheel-from-source]
--8<-- [start:pre-built-images]
--8<-- [end:pre-built-images]
--8<-- [start:build-image-from-source]
--8<-- [end:build-image-from-source]
--8<-- [start:supported-features]
vLLM-Metal provides:
- Native GPU acceleration using Metal
- MLX-based compute backend optimized for Apple Silicon
- OpenAI-compatible API server
- Support for popular model architectures
For specific feature support and limitations, refer to the [vLLM-Metal documentation](https://github.com/vllm-project/vllm-metal).
--8<-- [end:supported-features]
+32
View File
@@ -18,6 +18,10 @@ vLLM is a Python library that supports the following GPU variants. Select your G
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:installation"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:installation"
## Requirements
- OS: Linux
@@ -38,6 +42,10 @@ vLLM is a Python library that supports the following GPU variants. Select your G
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:requirements"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:requirements"
## Set up using Python
### Create a new Python environment
@@ -56,6 +64,10 @@ vLLM is a Python library that supports the following GPU variants. Select your G
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:set-up-using-python"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:set-up-using-python"
### Pre-built wheels {#pre-built-wheels}
=== "NVIDIA CUDA"
@@ -70,6 +82,10 @@ vLLM is a Python library that supports the following GPU variants. Select your G
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:pre-built-wheels"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:pre-built-wheels"
### Build wheel from source
=== "NVIDIA CUDA"
@@ -84,6 +100,10 @@ vLLM is a Python library that supports the following GPU variants. Select your G
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:build-wheel-from-source"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:build-wheel-from-source"
## Set up using Docker
### Pre-built images
@@ -102,6 +122,10 @@ vLLM is a Python library that supports the following GPU variants. Select your G
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:pre-built-images"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:pre-built-images"
--8<-- [end:pre-built-images]
### Build image from source
@@ -120,6 +144,10 @@ vLLM is a Python library that supports the following GPU variants. Select your G
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:build-image-from-source"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:build-image-from-source"
--8<-- [end:build-image-from-source]
## Supported features
@@ -135,3 +163,7 @@ vLLM is a Python library that supports the following GPU variants. Select your G
=== "Intel XPU"
--8<-- "docs/getting_started/installation/gpu.xpu.inc.md:supported-features"
=== "Apple Silicon"
--8<-- "docs/getting_started/installation/gpu.apple.inc.md:supported-features"