[CI/Build][Hardware][NVIDIA] Add opt-in Rubin Docker builds (#53443)

Signed-off-by: Shang Wang <samshang.wang@mail.utoronto.ca>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Shang Wang
2026-08-27 16:06:44 +00:00
committed by GitHub
co-authored by mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent d21c5b50a1
commit 9650dc73e2
7 changed files with 429 additions and 24 deletions
@@ -354,8 +354,9 @@ DOCKER_BUILDKIT=1 docker build . \
!!! note
By default vLLM will build for all GPU types for widest distribution. If you are just building for the
current GPU type the machine is running on, you can add the argument `--build-arg torch_cuda_arch_list=""`
for vLLM to find the current GPU type and build for that.
current GPU type, you can add `--build-arg torch_cuda_arch_list=""` to delegate architecture selection
to PyTorch. This requires the GPU to be visible to the container build; standard Docker BuildKit builds
do not expose it and PyTorch instead falls back to its common architecture list.
If you are using Podman instead of Docker, you might need to disable SELinux labeling by
adding `--security-opt label=disable` when running `podman build` command to avoid certain [existing issues](https://github.com/containers/buildah/discussions/4184).
@@ -422,6 +423,86 @@ For (G)B300, we recommend using CUDA 13, as shown in the following command.
After setting up QEMU, you can use the `--platform "linux/arm64"` flag in your `docker build` command.
#### [Preview] Building vLLM's Docker Image from Source for NVIDIA Rubin GPU Architecture
At this time, the Rubin preview is available only to NVIDIA engineers and
partners with access to NVIDIA's internal prerequisites. Set
`INSTALL_RUBIN_PRERELEASE=true` to enable the Rubin build path. It requires a
`requirements/rubin-prerelease.txt` file containing pointers to NVIDIA's
internal prerelease packages, plus NVIDIA-internal CUDA 13.4 or CUDA 13.5
development images for `BUILD_BASE_IMAGE` and `FINAL_BASE_IMAGE`.
Triton must currently be installed from source for Rubin compatibility.
Specify its repository with `TRITON_INSTALL_FROM_SOURCE_REPO`; an empty
`TRITON_INSTALL_FROM_SOURCE_REVISION` selects the repository's latest `main`,
while a commit, branch, or tag selects that revision. The tested revision
lowers SM107 through LLVM's SM100 target and uses the final CUDA image's
version-matched `ptxas` for SM107 assembly. This enables vLLM's default compiled
mode on VR200 and R100.
BuildKit does not automatically invalidate cached layers when a mutable Git
ref changes. Use `--no-cache-filter extensions-build` to refresh an empty,
branch, or tag revision.
Use `ARCH=arm64` on VR200 or `ARCH=amd64` on R100:
??? console "CUDA 13.4 command"
```bash
ARCH=arm64
docker buildx build --progress=plain --load \
--file docker/Dockerfile \
--target vllm-openai \
--platform "linux/${ARCH}" \
--tag "vllm/vllm-rubin-openai:prerelease-cu134-${ARCH}" \
--build-arg max_jobs="$(nproc)" \
--build-arg nvcc_threads=2 \
--build-arg RUN_WHEEL_CHECK=false \
--build-arg INSTALL_RUBIN_PRERELEASE=true \
--build-arg TRITON_INSTALL_FROM_SOURCE_REPO=https://github.com/triton-lang/triton.git \
--build-arg TRITON_INSTALL_FROM_SOURCE_REVISION=3f6e41132b5edf639bfb872ad73d4688765e08b8 \
--build-arg CUDA_VERSION=13.4.1 \
--build-arg BUILD_BASE_IMAGE="<cuda-13.4-manylinux-devel-image-${ARCH}>" \
--build-arg FINAL_BASE_IMAGE="<cuda-13.4-ubuntu24.04-devel-image-${ARCH}>" \
.
```
??? console "CUDA 13.5 command"
```bash
ARCH=arm64
docker buildx build --progress=plain --load \
--file docker/Dockerfile \
--target vllm-openai \
--platform "linux/${ARCH}" \
--tag "vllm/vllm-rubin-openai:prerelease-cu135-${ARCH}" \
--build-arg max_jobs="$(nproc)" \
--build-arg nvcc_threads=2 \
--build-arg RUN_WHEEL_CHECK=false \
--build-arg INSTALL_RUBIN_PRERELEASE=true \
--build-arg TRITON_INSTALL_FROM_SOURCE_REPO=https://github.com/triton-lang/triton.git \
--build-arg TRITON_INSTALL_FROM_SOURCE_REVISION=3f6e41132b5edf639bfb872ad73d4688765e08b8 \
--build-arg CUDA_VERSION=13.5.0 \
--build-arg BUILD_BASE_IMAGE="<cuda-13.5-manylinux-devel-image-${ARCH}>" \
--build-arg FINAL_BASE_IMAGE="<cuda-13.5-ubuntu24.04-devel-image-${ARCH}>" \
.
```
CUDA 13.5 compiled-mode qualification passed with NVIDIA driver versions
615.62.03 and 620.05. Driver 610.47.04 failed the tested compiled CUDA 13.5
kernel. Validate the exact driver version on the target system.
!!! note
Keep the default explicit `torch_cuda_arch_list`. GPU-less BuildKit builds
cannot inspect the host GPU. R100 and VR200 report compute capability 10.7,
for which the generic `10.0` target provides family-compatible kernels.
The Ubuntu `devel` final image is also required: the corresponding `base`
image lacks the CUDA runtime/JIT package closure used by vLLM and the
prerelease PyTorch wheel.
`RUN_WHEEL_CHECK=false` disables only the PyPI publication-size guard for
this private staging image.
#### Use the custom-built vLLM Docker image**
To run vLLM with the custom-built Docker image: