[Doc] Host model recipes in recipes/ (#61302)

Signed-off-by: esmeetu <jasonailu87@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Roy Wang
2026-10-17 09:14:36 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent f98ae6a9fd
commit b515648641
193 changed files with 44859 additions and 6 deletions
@@ -0,0 +1,178 @@
meta:
title: "Phi-4"
slug: "phi-4"
provider: "Microsoft"
description: "Microsoft's Phi-4 family of lightweight dense models (mini-instruct, reasoning, multimodal) with 128K context"
date_added: 2026-04-02
date_updated: 2026-04-17
tasks:
- text
- multimodal
related_recipes: []
hardware:
h100: verified
model:
model_id: "microsoft/Phi-4-mini-instruct"
min_vllm_version: "0.7.0"
architecture: dense
parameter_count: "4B"
active_parameters: "4B"
context_length: 131072
base_args: []
base_env: {}
features:
text_only:
description: "Skip loading the vision encoder for text-only workloads — frees VRAM for KV cache. Mutually exclusive with encoder_parallel."
args:
- "--language-model-only"
encoder_parallel:
description: "Run the vision encoder in data-parallel mode — avoids TP comm overhead on the small encoder. Mutually exclusive with text_only."
args:
- "--mm-encoder-tp-mode"
- "data"
opt_in_features:
- text_only
- encoder_parallel
variants:
default:
precision: bf16
vram_minimum_gb: 10
description: "Phi-4-mini-instruct, conversational instruction-tuned"
mini_reasoning:
model_id: "microsoft/Phi-4-mini-reasoning"
precision: bf16
vram_minimum_gb: 10
description: "Optimized for reasoning tasks"
reasoning:
model_id: "microsoft/Phi-4-reasoning"
precision: bf16
vram_minimum_gb: 30
description: "Advanced reasoning capabilities (14B)"
multimodal:
model_id: "microsoft/Phi-4-multimodal-instruct"
min_vllm_version: "0.8.0"
precision: bf16
vram_minimum_gb: 16
description: "Multimodal instruction-following (text + image)"
extra_args:
- "--trust-remote-code"
fp8:
model_id: "RedHatAI/Phi-4-mini-instruct-FP8-dynamic"
precision: fp8
vram_minimum_gb: 5
description: "RedHatAI FP8 (E4M3) with dynamic per-token activation quantization"
extra_args:
- "--kv-cache-dtype"
- "fp8"
compatible_strategies:
- single_node_tp
- multi_node_tp
hardware_overrides: {}
strategy_overrides: {}
guide: |
## Overview
The Phi-4 family includes several lightweight, open models from Microsoft. These models
can process text and, in some variants, multimodal inputs like images, to generate
text outputs. They come with a 128K token context length.
## Prerequisites
- Hardware: 1x GPU with >=16 GB VRAM (A100, L40S, H100, etc.)
- vLLM >= 0.7.0 (the `multimodal` variant serves `Phi-4-multimodal-instruct`, which needs vLLM >= 0.8.0)
### Install vLLM
```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```
## Launch commands
Phi-4-mini-instruct on a single GPU:
```bash
vllm serve microsoft/Phi-4-mini-instruct \
--host 0.0.0.0 \
--max-model-len 4000
```
Phi-4-multimodal-instruct (requires --trust-remote-code for LoRA modules):
```bash
vllm serve microsoft/Phi-4-multimodal-instruct \
--host 0.0.0.0 \
--max-model-len 4000 \
--trust-remote-code
```
## Client Usage
```python
from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1", timeout=3600)
response = client.chat.completions.create(
model="microsoft/Phi-4-mini-instruct",
messages=[{"role": "user", "content": "Write a short story"}],
temperature=0.0,
)
print(response.choices[0].message.content)
```
Multimodal (requires Phi-4-multimodal-instruct):
```python
response = client.chat.completions.create(
model="microsoft/Phi-4-multimodal-instruct",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Describe this image."},
{"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}},
],
}],
)
```
## Available Phi-4 Variants
- `microsoft/Phi-4-mini-instruct` — conversational instruction-tuned
- `microsoft/Phi-4-mini-reasoning` — optimized for reasoning
- `microsoft/Phi-4-reasoning` — advanced reasoning
- `microsoft/Phi-4-multimodal-instruct` — multimodal (text + image)
## Benchmarking
```bash
vllm bench serve \
--model microsoft/Phi-4-mini-instruct \
--dataset-name random \
--random-input-len 2000 \
--random-output-len 512 \
--num-prompts 100
```
## Troubleshooting
- Multimodal variant fails to load: add `--trust-remote-code`.
## References
- [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct)
- [Phi-4-multimodal-instruct](https://huggingface.co/microsoft/Phi-4-multimodal-instruct)
- [Phi-4-reasoning](https://huggingface.co/microsoft/Phi-4-reasoning)
@@ -0,0 +1,103 @@
meta:
title: "Phi-4-multimodal-instruct"
slug: "phi-4-multimodal-instruct"
provider: "Microsoft"
description: "Microsoft Phi-4 multimodal model supporting text, image, and audio inputs with text outputs."
date_added: 2026-08-19
date_updated: 2026-09-12
tasks:
- multimodal
performance_headline: "Validated on Intel Xeon 6"
hardware:
xeon6: verified
model:
model_id: "microsoft/Phi-4-multimodal-instruct"
min_vllm_version: "0.8.0"
architecture: dense
parameter_count: "5.6B"
active_parameters: "5.6B"
context_length: 131072
base_args:
- "--trust-remote-code"
base_env: {}
features: {}
opt_in_features: []
variants:
default:
precision: bf16
vram_minimum_gb: 14
description: "Full precision BF16"
supported_hardware: [xeon6]
compatible_strategies:
- single_node_tp
- multi_node_tp
hardware_overrides: {}
strategy_overrides: {}
guide: |
## Overview
Phi-4-multimodal-instruct processes text, image, and audio inputs and
generates text. `--trust-remote-code` is retained because the model's
multimodal LoRA components require it.
## Prerequisites
- Hardware: Intel Xeon 6/Xeon 5 CPUs
- vLLM >= 0.8.0
### Docker (Intel Xeon 6 CPUs)
```bash
docker pull vllm/vllm-openai-cpu:latest-x86_64
```
## Intel Xeon 6
```bash
vllm serve microsoft/Phi-4-multimodal-instruct \
--trust-remote-code \
--tensor-parallel-size 1
```
Docker (the image entrypoint is `vllm serve`):
```bash
docker run \
--privileged --ipc=host -p 8000:8000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai-cpu:latest-x86_64 microsoft/Phi-4-multimodal-instruct \
--trust-remote-code \
--tensor-parallel-size 1
```
The example uses TP=1. Adjust TP/DP for the deployment topology;
host-specific CPU placement is intentionally not hard-coded.
## Runtime and Platform Tuning
The following settings are intentionally not prescribed as portable CPU model
defaults because they depend on the workload, hardware, or runtime environment:
- `--max-num-batched-tokens`: scheduler/throughput tuning.
- `--max-num-seqs`: concurrency and scheduler-capacity tuning.
- `--gpu-memory-utilization`: platform memory-budget tuning.
- `--no-enable-prefix-caching`: workload/benchmark cache-policy tuning.
- `VLLM_ENGINE_ITERATION_TIMEOUT_S`: operational runtime timeout.
These settings may still appear in validated hardware-specific overrides. Tune
them at deployment time based on platform resources, workload shape, and
latency/throughput goals.
## References
- [Model card](https://huggingface.co/microsoft/Phi-4-multimodal-instruct)
- [vLLM Phi-4 usage guide](https://docs.vllm.ai/projects/recipes/en/stable/Microsoft/Phi-4.html)
- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)
@@ -0,0 +1,98 @@
meta:
title: "Phi-4-reasoning"
slug: "phi-4-reasoning"
provider: "Microsoft"
description: "Microsoft Phi-4 14B dense reasoning model with 32K context."
date_added: 2026-08-19
date_updated: 2026-09-12
tasks:
- text
performance_headline: "Validated on Intel Xeon 6 with BF16 serving"
hardware:
xeon6: verified
model:
model_id: "microsoft/Phi-4-reasoning"
min_vllm_version: "0.4.2"
architecture: dense
parameter_count: "14B"
active_parameters: "14B"
context_length: 32768
base_args: []
base_env: {}
features: {}
opt_in_features: []
variants:
default:
precision: bf16
vram_minimum_gb: 34
description: "Full precision BF16"
supported_hardware: [xeon6]
compatible_strategies:
- single_node_tp
- multi_node_tp
hardware_overrides: {}
strategy_overrides: {}
guide: |
## Overview
Phi-4-reasoning is Microsoft's 14B dense reasoning model. Intel Xeon 6 CPU
serving was validated with BF16.
## Prerequisites
- Hardware: Intel Xeon 6/Xeon 5 CPUs
- vLLM >= 0.4.2
### Docker (Intel Xeon 6 CPUs)
```bash
docker pull vllm/vllm-openai-cpu:latest-x86_64
```
## Intel Xeon 6
```bash
vllm serve microsoft/Phi-4-reasoning \
--tensor-parallel-size 1
```
Docker (the image entrypoint is `vllm serve`):
```bash
docker run \
--privileged --ipc=host -p 8000:8000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
vllm/vllm-openai-cpu:latest-x86_64 microsoft/Phi-4-reasoning \
--tensor-parallel-size 1
```
The example uses TP=1. Adjust TP/DP for the deployment topology;
host-specific CPU placement is intentionally not hard-coded.
## Runtime and Platform Tuning
The following settings are intentionally not prescribed as portable CPU model
defaults because they depend on the workload, hardware, or runtime environment:
- `--max-num-batched-tokens`: scheduler/throughput tuning.
- `--max-num-seqs`: concurrency and scheduler-capacity tuning.
- `--gpu-memory-utilization`: platform memory-budget tuning.
- `--no-enable-prefix-caching`: workload/benchmark cache-policy tuning.
- `VLLM_ENGINE_ITERATION_TIMEOUT_S`: operational runtime timeout.
These settings may still appear in validated hardware-specific overrides. Tune
them at deployment time based on platform resources, workload shape, and
latency/throughput goals.
## References
- [Model card](https://huggingface.co/microsoft/Phi-4-reasoning)
- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)