connects vLLM Recipes with vLLM's native config-based deployment and benchmark (#51308)

Signed-off-by: louie-tsai <louie.tsai@intel.com>
This commit is contained in:
Louie Tsai
2026-08-11 13:53:51 +00:00
committed by GitHub
parent f863387250
commit 5b184f775a
6 changed files with 773 additions and 43 deletions
+21
View File
@@ -8,6 +8,27 @@ toc_depth: 2
--8<-- "docs/getting_started/installation/gpu.md:pre-built-images"
## Run a vLLM Recipes configuration
[vLLM Recipes](https://recipes.vllm.ai/) can be converted into `config.yaml`
and `env.sh`. See the
[Recipes conversion tool README](../../tools/recipes/README.md) for usage.
For Docker, mount both files and source `env.sh` inside the container before
starting vLLM:
```bash
docker run --rm --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-v "$PWD/config.yaml:/recipe/config.yaml:ro" \
-v "$PWD/env.sh:/recipe/env.sh:ro" \
-p 8000:8000 \
--ipc=host \
--entrypoint /bin/bash \
vllm/vllm-openai:latest \
-lc 'source /recipe/env.sh && exec vllm serve --config /recipe/config.yaml'
```
## Persist the compile cache across containers
Mounting the Hugging Face cache keeps model weights across containers, but each