diff --git a/docker/Dockerfile.cpu b/docker/Dockerfile.cpu index ad36c18d49..ed5dfa5d24 100644 --- a/docker/Dockerfile.cpu +++ b/docker/Dockerfile.cpu @@ -482,5 +482,6 @@ LABEL ai.vllm.build.python-version="${PYTHON_VERSION:-3.12}" # stage above adds examples/ for testing only, so without this the published # vllm-openai-cpu image would not ship examples/*.jinja. COPY examples examples +COPY tools/recipes tools/recipes ENTRYPOINT ["vllm", "serve"] diff --git a/docs/getting_started/installation/cpu.x86.inc.md b/docs/getting_started/installation/cpu.x86.inc.md index 260ffe2863..ebae441f93 100644 --- a/docs/getting_started/installation/cpu.x86.inc.md +++ b/docs/getting_started/installation/cpu.x86.inc.md @@ -183,6 +183,40 @@ docker run \ vllm/vllm-openai-cpu:latest-x86_64 ``` +### Serve with vLLM Recipes + +The CPU image includes the [vLLM Recipes](https://recipes.vllm.ai/) deployment +tool. Recipe-assisted serving queries the site at runtime, so the image can use +the latest published recipe for the selected model and hardware. + +To find the value for `--hardware`, open vLLM Recipes, select the model, and +choose the target in the **Hardware** picker. The selected page URL also +contains the hardware key; for example, `?hardware=xeon6` maps to +`--hardware xeon6`. + +Keep the normal image entrypoint for regular serving. For recipe-assisted +serving, override it with `serve_with_recipe.sh`: + +```bash +docker run --rm \ + --shm-size=4g \ + -v ~/.cache/huggingface:/root/.cache/huggingface \ + -p 8000:8000 \ + --env "HF_TOKEN=" \ + --entrypoint tools/recipes/serve_with_recipe.sh \ + vllm/vllm-openai-cpu:latest-x86_64 \ + --model meta-llama/Llama-3.1-8B-Instruct \ + --hardware xeon6 +``` + +For Xeon 6, the script enables hardware detection automatically, generates +`config.yml` and `env.sh`, and starts `vllm serve`. The image keeps +`/vllm-workspace` as its working directory so relative recipe assets under +`examples/` continue to resolve. + +See the [Recipes tool documentation](../../../tools/recipes/README.md) for +interactive discovery and additional options. + --8<-- [end:pre-built-images] --8<-- [start:build-image-from-source] diff --git a/docs/models/hardware_supported_models/cpu.md b/docs/models/hardware_supported_models/cpu.md index 10ca8caa0e..6e6da02876 100644 --- a/docs/models/hardware_supported_models/cpu.md +++ b/docs/models/hardware_supported_models/cpu.md @@ -12,17 +12,17 @@ ## Deploy from a vLLM Recipe -The **Recipe** column below links to validated Xeon 6 configurations when -available. Use the Recipe conversion tool to generate `config.yaml` and -`env.sh`; see the -[Recipes conversion tool README](../../../tools/recipes/README.md) for usage. +The **Recipe** column below links to published Xeon 6 configurations when +available. Open a recipe to review the latest deployment configuration for the +model and hardware. -Load the generated environment before starting vLLM: +For one-step deployment with the pre-built CPU image, see +[Serve with vLLM Recipes](../../getting_started/installation/cpu.md#serve-with-vllm-recipes). +The image includes the Recipes tool, which can retrieve the latest published +recipe, apply Xeon hardware detection, and start `vllm serve`. -```bash -source env.sh -vllm serve --config config.yaml -``` +For converter usage and interactive model/hardware discovery, see the +[Recipes tool documentation](../../../tools/recipes/README.md). ## Recommended Models diff --git a/tools/recipes/README.md b/tools/recipes/README.md index 253bd7a726..90ad33b3eb 100644 --- a/tools/recipes/README.md +++ b/tools/recipes/README.md @@ -23,8 +23,9 @@ refine the initial configuration. Sweep tuning is an optional validation step. ## Getting Started -Use `serve_with_recipe.sh` to generate the recipe configuration and start vLLM -in one step: +Use `serve_with_recipe.sh` to fetch the selected deployment from +[vLLM Recipes](https://recipes.vllm.ai/), generate the configuration, and start +vLLM in one step: ```bash tools/recipes/serve_with_recipe.sh \ @@ -32,6 +33,23 @@ tools/recipes/serve_with_recipe.sh \ --hardware xeon6 ``` +The converter queries `recipes.vllm.ai` at runtime, so it can use the latest +published recipe for the requested model and hardware without rebuilding the +vLLM image. + +### Find the hardware name + +Open [vLLM Recipes](https://recipes.vllm.ai/), select the model, and choose the +target in the **Hardware** picker. The selected page URL contains the hardware +key. For example, `?hardware=xeon6` maps to `--hardware xeon6`. + +You can also run the converter without arguments for interactive model, +hardware, and strategy discovery: + +```bash +python3 tools/recipes/recipe_json_to_vllm_config.py +``` + For `xeon6`, the script enables hardware detection automatically. Generated `config.yml` and `env.sh` files remain in the directory where the script was invoked.