From 2e860de498a229018d8795afa34417f43b813d94 Mon Sep 17 00:00:00 2001 From: Nils Matteson Date: Sat, 25 Jul 2026 23:24:01 -0600 Subject: [PATCH] [Doc] Add compile cache volume example to the Docker deployment page (#49782) Signed-off-by: Nils Matteson --- docs/deployment/docker.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/deployment/docker.md b/docs/deployment/docker.md index a8debf2cdb..b5f3a8b049 100644 --- a/docs/deployment/docker.md +++ b/docs/deployment/docker.md @@ -8,6 +8,26 @@ toc_depth: 2 --8<-- "docs/getting_started/installation/gpu.md:pre-built-images" +## Persist the compile cache across containers + +Mounting the Hugging Face cache keeps model weights across containers, but each +new container still starts with an empty `VLLM_CACHE_ROOT` (default +`~/.cache/vllm`) and recompiles the model's `torch.compile` artifacts. Mount a +named volume at that path to reuse the inductor, Triton, and AOT artifacts from +the second container onward: + +```bash +docker run --rm --gpus all \ + -v ~/.cache/huggingface:/root/.cache/huggingface \ + -v vllm-cache:/root/.cache/vllm \ + -p 8000:8000 \ + vllm/vllm-openai:latest \ + meta-llama/Llama-3.1-8B-Instruct +``` + +See [Faster Startup](../configuration/optimization.md#faster-startup) for the +mechanism and for what invalidates the cache. + ## Run as a non-root user The CUDA `vllm/vllm-openai` image runs as root by default for backward