[Docs] Add model recipes skill (#57786)
This commit is contained in:
@@ -0,0 +1,151 @@
|
|||||||
|
---
|
||||||
|
name: model-recipes
|
||||||
|
description: Find vLLM deployment recipes and configs for LLMs and Hugging Face models by model and hardware. Use for recipe lookup, creation, and updates.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Model Recipes
|
||||||
|
|
||||||
|
Model serving recipes are maintained in the
|
||||||
|
[models/ directory of vllm-project/recipes](https://github.com/vllm-project/recipes/tree/main/models).
|
||||||
|
The published catalog is at
|
||||||
|
[recipes.vllm.ai](https://recipes.vllm.ai).
|
||||||
|
|
||||||
|
## Usage notes
|
||||||
|
|
||||||
|
1. Recipe lookup returns commands and deployment guidance without launching
|
||||||
|
`vllm serve`.
|
||||||
|
2. Use published configurations or the recipes repository's command generator
|
||||||
|
for supported selections. Return commands as published, including their
|
||||||
|
flags and parallelism settings.
|
||||||
|
3. Recipes are starting points for deployment and may not be tuned for your
|
||||||
|
workload. Check each recipe for its verified configurations, and use
|
||||||
|
benchmarks of your workload to guide further tuning.
|
||||||
|
4. To create or update a recipe, follow the
|
||||||
|
[authoring workflow below](#create-or-update-a-recipe).
|
||||||
|
|
||||||
|
## Find a recipe
|
||||||
|
|
||||||
|
Read recipes from the published JSON on recipes.vllm.ai. It is generated from
|
||||||
|
the source YAML with variant, hardware, and strategy settings already
|
||||||
|
resolved; the YAML contains comments and conditional overrides, so do not use
|
||||||
|
it for lookup. When presenting results, link the website page. Read the source
|
||||||
|
YAML only when creating or updating a recipe.
|
||||||
|
|
||||||
|
Each recipe has a page at `https://recipes.vllm.ai/<org>/<repo>` and JSON at
|
||||||
|
the same path plus `.json`. Paths use the full Hugging Face ID, preserving
|
||||||
|
case; do not substitute `meta.slug` or a display title. For example,
|
||||||
|
`Qwen/Qwen3.8-27B` maps to
|
||||||
|
[recipes.vllm.ai/Qwen/Qwen3.8-27B](https://recipes.vllm.ai/Qwen/Qwen3.8-27B).
|
||||||
|
|
||||||
|
Recipe JSON is often over 20 KB, mostly the `guide`. Instead of printing it,
|
||||||
|
read only what the request needs: each step below returns the paths for the
|
||||||
|
next, so stop once the request is answered. Commands need `curl` and `jq`.
|
||||||
|
[scripts/read_recipe.sh](scripts/read_recipe.sh) is in this skill's directory;
|
||||||
|
the examples use its path from the vLLM repository root.
|
||||||
|
|
||||||
|
### Search the catalog
|
||||||
|
|
||||||
|
Skip this step if the exact HF ID is known. Otherwise search by model ID,
|
||||||
|
family, or provider using case-insensitive keywords. Each
|
||||||
|
whitespace-separated word must occur in the ID, title, or provider:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsSL https://recipes.vllm.ai/models.json |
|
||||||
|
jq --arg q 'qwen 27b' '
|
||||||
|
($q | ascii_downcase | [scan("\\S+")]) as $words | .[] |
|
||||||
|
([.hf_id, .title, .provider] | join(" ") | ascii_downcase) as $text |
|
||||||
|
select(all($words[]; . as $word | $text | contains($word))) |
|
||||||
|
{hf_id, json, url, derived_from}'
|
||||||
|
```
|
||||||
|
|
||||||
|
`json` is the recipe data path; `url` is its page path. Prefix both with
|
||||||
|
`https://recipes.vllm.ai`. A `derived_from` entry is a variant of a parent
|
||||||
|
recipe: fetch its own `json` to get that checkpoint's command. Its page may
|
||||||
|
belong to the parent, so link the returned `url` rather than assuming every
|
||||||
|
checkpoint has a separate page.
|
||||||
|
|
||||||
|
If there is no exact match, search the family and inspect `variants` before
|
||||||
|
concluding the recipe is missing.
|
||||||
|
|
||||||
|
### Summarize the recipe
|
||||||
|
|
||||||
|
Save the recipe JSON to a file and summarize it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
read_recipe=.agents/skills/model-recipes/scripts/read_recipe.sh
|
||||||
|
recipe=$(mktemp)
|
||||||
|
curl -fsSL https://recipes.vllm.ai/Qwen/Qwen3.8-27B.json -o "$recipe"
|
||||||
|
"$read_recipe" "$recipe"
|
||||||
|
```
|
||||||
|
|
||||||
|
`default` is the recommended hardware, strategy, and variant; `hardware` maps
|
||||||
|
each hardware ID to its command path; `recipe_verified_hardware` lists hardware
|
||||||
|
where the recipe as a whole was verified end to end. For capability
|
||||||
|
requirements, such as tool calling or text-to-video, check `features` and
|
||||||
|
`omni_tasks`, then confirm in the guide.
|
||||||
|
Some recipes, including Omni, lack `hardware`; skip hardware selection and
|
||||||
|
use the guide's launch and task-specific sections.
|
||||||
|
|
||||||
|
### Select hardware and strategy
|
||||||
|
|
||||||
|
Match a hardware ID to the user's GPU and fetch its path from `hardware`,
|
||||||
|
prefixed with `https://recipes.vllm.ai`, rather than constructing a URL. That
|
||||||
|
response is the command object itself: single-node objects carry `command`,
|
||||||
|
and multi-node objects carry `head_command` and `worker_commands` instead, so
|
||||||
|
list its keys before selecting fields.
|
||||||
|
Check `hardware_profile` to confirm that the selected configuration matches
|
||||||
|
the user's GPU model and specifications. Its `gpu_count` describes the profile,
|
||||||
|
not necessarily the number of GPUs used by the command. Check the command's
|
||||||
|
parallelism settings, `node_count`, and all `worker_commands` against the
|
||||||
|
user's available topology, and compare the variant's `vram_minimum_gb`, an
|
||||||
|
estimate, with the total memory of the GPUs the command uses; a configuration
|
||||||
|
below that minimum is not a working option unless the guide says it was
|
||||||
|
tested. Ask when the user's hardware is unclear. If no
|
||||||
|
published command or guide command fits, report the mismatch and return the
|
||||||
|
closest one unchanged; list any adjustments the user's setup would need
|
||||||
|
separately and mark them unverified instead of editing the command.
|
||||||
|
|
||||||
|
For another strategy, follow `alternatives` in the same response. These are
|
||||||
|
static files; query parameters do not change their configuration. For other
|
||||||
|
feature/topology selections, use the recipes repository's command synthesis.
|
||||||
|
|
||||||
|
### Read deployment guidance
|
||||||
|
|
||||||
|
Read the guide sections that apply to the selection, such as prerequisites,
|
||||||
|
launch notes for the selected hardware or variant, troubleshooting, and known
|
||||||
|
limitations. Pass entries from `guide_sections` or prefixes of their text.
|
||||||
|
The script prints every matching section with its subsections and fails when
|
||||||
|
a section matches nothing:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
"$read_recipe" "$recipe" prerequisites troubleshooting
|
||||||
|
```
|
||||||
|
|
||||||
|
Query other fields with `jq` on the saved file, such as `features.<name>`,
|
||||||
|
`hardware_overrides`, or `omni.tasks` for request examples.
|
||||||
|
|
||||||
|
### Return the result
|
||||||
|
|
||||||
|
Return the website link, selected checkpoint and hardware, the requested
|
||||||
|
command, and relevant prerequisites and caveats from the guide. Pick one
|
||||||
|
command and copy it, including its Docker form, exactly as the command object
|
||||||
|
or guide section gives it; do not combine commands from different sections or
|
||||||
|
add, drop, or reorder flags. List optional changes separately.
|
||||||
|
|
||||||
|
For each command, state its source, either a generated hardware or strategy
|
||||||
|
configuration or a named guide section, and whether that source says it was
|
||||||
|
tested. A `recipe_verified_hardware` entry covers the recipe on that
|
||||||
|
hardware, not every variant, strategy, or guide command, so it does not make a
|
||||||
|
command verified. Generated configurations are not automatically verified.
|
||||||
|
|
||||||
|
## Create or update a recipe
|
||||||
|
|
||||||
|
Recipes cover official or widely used checkpoints, with configurations verified
|
||||||
|
end to end on real hardware. Do not add recipes for arbitrary community
|
||||||
|
uploads, untested hardware claims, or non-standard flags that only work for one
|
||||||
|
setup. Before opening a pull request, confirm that the user ran the
|
||||||
|
configuration and can share the results.
|
||||||
|
|
||||||
|
Follow the recipes repository's
|
||||||
|
[add-recipe skill](https://github.com/vllm-project/recipes/blob/main/.agents/skills/add-recipe/SKILL.md)
|
||||||
|
in a checkout of `vllm-project/recipes` or the user's intended fork.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
interface:
|
||||||
|
display_name: "Model Recipes"
|
||||||
|
short_description: "Find vLLM deployment recipes and configs for LLMs and Hugging Face models by model and hardware."
|
||||||
|
default_prompt: "Use $model-recipes to find a recipe for my model and hardware, and return its commands and deployment caveats."
|
||||||
+66
@@ -0,0 +1,66 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Usage: read_recipe.sh <recipe.json> [guide-section...]
|
||||||
|
# Without sections, print a compact summary of a recipes.vllm.ai recipe JSON.
|
||||||
|
# With sections, print each guide section whose heading starts with one of
|
||||||
|
# them (case-insensitive), including its subsections.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
if [ $# -lt 1 ]; then
|
||||||
|
echo "Usage: $0 <recipe.json> [guide-section...]" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Print "<level>\t<heading text>\t<line>" for guide lines, marking headings
|
||||||
|
# outside code fences with their level and other lines with 0.
|
||||||
|
guide_lines() {
|
||||||
|
jq -r '.guide // ""' "$1" | awk '
|
||||||
|
/^[[:space:]]*(```|~~~)/ { fence = !fence }
|
||||||
|
!fence && match($0, /^#+ /) {
|
||||||
|
print RLENGTH - 1 "\t" substr($0, RLENGTH + 1) "\t" $0
|
||||||
|
next
|
||||||
|
}
|
||||||
|
{ print "0\t\t" $0 }'
|
||||||
|
}
|
||||||
|
|
||||||
|
if [ $# -gt 1 ]; then
|
||||||
|
recipe=$1
|
||||||
|
shift
|
||||||
|
status=0
|
||||||
|
for section in "$@"; do
|
||||||
|
guide_lines "$recipe" | awk -F '\t' -v h="$section" '
|
||||||
|
BEGIN { h = tolower(h); sub(/^#+[[:space:]]*/, "", h) }
|
||||||
|
$1 > 0 && lvl && $1 <= lvl { lvl = 0 }
|
||||||
|
$1 > 1 && !lvl && index(tolower($2), h) == 1 { lvl = $1; found = 1 }
|
||||||
|
lvl { sub(/^[^\t]*\t[^\t]*\t/, ""); print }
|
||||||
|
END {
|
||||||
|
if (!found) {
|
||||||
|
print "No guide section starts with: " h > "/dev/stderr"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
}' || status=1
|
||||||
|
done
|
||||||
|
exit $status
|
||||||
|
fi
|
||||||
|
|
||||||
|
sections=$(guide_lines "$1" | awk -F '\t' '$1 == 2 || $1 == 3 { print $3 }' |
|
||||||
|
jq -R . | jq -s .)
|
||||||
|
|
||||||
|
jq --argjson sections "$sections" '{
|
||||||
|
hf_id,
|
||||||
|
title: .meta.title,
|
||||||
|
tasks: .meta.tasks,
|
||||||
|
recipe_verified_hardware: .meta.hardware,
|
||||||
|
omni_tasks: (.omni.tasks // null |
|
||||||
|
if . then map(objects |= {id, label, endpoint, model_id, description})
|
||||||
|
else null end),
|
||||||
|
variants: (.variants // {} | map_values(
|
||||||
|
{model_id, precision, vram_minimum_gb, supported_hardware} |
|
||||||
|
with_entries(select(.value != null)))),
|
||||||
|
features: (.features // {} | keys),
|
||||||
|
opt_in_features,
|
||||||
|
default: (.recommended_command // null |
|
||||||
|
if . then {hardware, strategy, variant} else null end),
|
||||||
|
hardware: .recommended_command.by_hardware,
|
||||||
|
guide_sections: $sections
|
||||||
|
} | with_entries(select(.value != null))' "$1"
|
||||||
Symlink
+1
@@ -0,0 +1 @@
|
|||||||
|
../../.agents/skills/model-recipes
|
||||||
Reference in New Issue
Block a user