diff --git a/README.md b/README.md index 42777436c6..ca31a8cce9 100644 --- a/README.md +++ b/README.md @@ -48,7 +48,7 @@ vLLM is flexible and easy to use with: - Tool calling and reasoning parsers - OpenAI-compatible API server, plus Anthropic Messages API and gRPC support - Efficient multi-LoRA support for dense and MoE layers -- Support for NVIDIA GPUs, AMD GPUs, and x86/ARM/PowerPC CPUs. Additionally, diverse hardware plugins such as Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, MetaX GPU, and more. +- Support for NVIDIA GPUs, AMD GPUs, Intel GPUs, and x86/ARM/PowerPC CPUs. Additionally, diverse hardware plugins such as Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, MetaX GPU, and more. vLLM seamlessly supports 200+ model architectures on Hugging Face, including: diff --git a/docs/getting_started/installation/gpu.xpu.inc.md b/docs/getting_started/installation/gpu.xpu.inc.md index 8564f2a726..e0badd7136 100644 --- a/docs/getting_started/installation/gpu.xpu.inc.md +++ b/docs/getting_started/installation/gpu.xpu.inc.md @@ -58,7 +58,40 @@ VLLM_TARGET_DEVICE=xpu pip install --no-build-isolation -e . -v --8<-- [end:build-wheel-from-source] --8<-- [start:pre-built-images] -Currently, we release prebuilt XPU images at docker [hub](https://hub.docker.com/r/intel/vllm/tags) based on vLLM released version. For more information, please refer release [note](https://github.com/intel/ai-containers/blob/main/vllm). +vLLM offers official Docker images for deployment. +The images can be used to run OpenAI compatible server and are available on Docker Hub as [vllm/vllm-openai-xpu](https://hub.docker.com/r/vllm/vllm-openai-xpu/tags). + +- `vllm/vllm-openai-xpu:latest` — stable release, available starting from v0.26.0 +- `vllm/vllm-openai-xpu:nightly` — preview build from the latest development branch, use this if you want the latest features and fixes + +```bash +docker run --rm \ + --network=host \ + --device /dev/dri:/dev/dri \ + -v /dev/dri/by-path:/dev/dri/by-path \ + -v ~/.cache/huggingface:/root/.cache/huggingface \ + --env "HF_TOKEN=$HF_TOKEN" \ + --ipc=host \ + --privileged \ + vllm/vllm-openai-xpu: \ + --model Qwen/Qwen3-0.6B +``` + +To use the docker image as base for development, you can launch it in interactive session through overriding the entrypoint. + +???+ console "Commands" + ```bash + docker run --rm -it \ + --network=host \ + --device /dev/dri:/dev/dri \ + -v /dev/dri/by-path:/dev/dri/by-path \ + -v ~/.cache/huggingface:/root/.cache/huggingface \ + --env "HF_TOKEN=$HF_TOKEN" \ + --ipc=host \ + --privileged \ + --entrypoint /bin/bash \ + vllm/vllm-openai-xpu: + ``` --8<-- [end:pre-built-images] --8<-- [start:build-image-from-source] diff --git a/docs/getting_started/quickstart.md b/docs/getting_started/quickstart.md index f674853f81..8a15a669c4 100644 --- a/docs/getting_started/quickstart.md +++ b/docs/getting_started/quickstart.md @@ -65,6 +65,15 @@ This guide will help you quickly get started with vLLM to perform: !!! tip A nightly Docker image is also available as [vllm/vllm-openai-rocm:nightly](https://hub.docker.com/r/vllm/vllm-openai-rocm/tags) for testing the latest development builds. +=== "Intel GPU" + + vLLM supports Intel GPUs through the XPU backend. Pre-built XPU wheels will be available soon. + + Official Docker images for Intel GPUs are added to the vLLM release starting from v0.26.0. Nightly Docker image is also available as [vllm/vllm-openai-xpu:nightly](https://hub.docker.com/r/vllm/vllm-openai-xpu/tags). + + !!! tip + For more detailed instructions, including building from source and Docker image setup, please refer to the [GPU installation guide](installation/gpu.md) and select the "Intel XPU" tab. + === "Google TPU" To run vLLM on Google TPUs, you need to install the `vllm-tpu` package.