From cc40c3673b47a58d1326d1e7a2798f1f67a94a8f Mon Sep 17 00:00:00 2001 From: Louie Tsai Date: Sun, 23 Aug 2026 23:42:28 -0700 Subject: [PATCH] [Xeon][doc]add Xeon recipes into table (#53226) Signed-off-by: louie-tsai --- docs/models/hardware_supported_models/cpu.md | 47 ++++++++++---------- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/docs/models/hardware_supported_models/cpu.md b/docs/models/hardware_supported_models/cpu.md index f80c6355b4..6be3c3bd3d 100644 --- a/docs/models/hardware_supported_models/cpu.md +++ b/docs/models/hardware_supported_models/cpu.md @@ -30,38 +30,39 @@ vllm serve --config config.yaml | Model | Architecture | Supported | Recipe | | ------------------------------------ | ---------------------------------------- | --------- | ------ | -| unsloth/gpt-oss-20b | GptOssForCausalLM | ✅ | — | | openai/gpt-oss-20b | GptOssForCausalLM | ✅ | — | +| meta-llama/Llama-3.1-8B | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B?hardware=xeon6) | | meta-llama/Llama-3.1-8B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct?hardware=xeon6) | -| meta-llama/Llama-3.2-1B | LlamaForCausalLM | ✅ | — | -| meta-llama/Llama-3.2-3B-Instruct | LlamaForCausalLM | ✅ | — | +| meta-llama/Llama-3.2-1B | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B?hardware=xeon6) | +| meta-llama/Llama-3.2-1B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B-Instruct?hardware=xeon6) | +| meta-llama/Llama-3.2-3B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-3B-Instruct?hardware=xeon6) | | meta-llama/Llama-3.3-70B-Instruct | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.3-70B-Instruct?hardware=xeon6) | -| RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8 | LlamaForCausalLM | ✅ | — | -| RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | — | -| RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | — | -| RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | — | +| RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B?hardware=xeon6&variant=w8a8_int8) | +| RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct?hardware=xeon6&variant=w8a8_int8) | +| RedHatAI/Llama-3.2-1B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B-Instruct?hardware=xeon6&variant=w8a8_int8) | +| RedHatAI/Llama-3.2-3B-Instruct-quantized.w8a8 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-3B-Instruct?hardware=xeon6&variant=w8a8_int8) | | RedHatAI/DeepSeek-R1-Distill-Llama-70B-quantized.w8a8 | LlamaForCausalLM | ✅ | — | -| hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 | LlamaForCausalLM | ✅ | — | -| AMead10/Llama-3.2-1B-Instruct-AWQ | LlamaForCausalLM | ✅ | — | -| AMead10/Llama-3.2-3B-Instruct-AWQ | LlamaForCausalLM | ✅ | — | +| hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4 | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.1-8B-Instruct?hardware=xeon6&variant=awq_int4) | +| AMead10/Llama-3.2-1B-Instruct-AWQ | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-1B-Instruct?hardware=xeon6&variant=awq_int4) | +| AMead10/Llama-3.2-3B-Instruct-AWQ | LlamaForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/meta-llama/Llama-3.2-3B-Instruct?hardware=xeon6&variant=awq_int4) | | TheBloke/TinyLlama-1.1B-Chat-v1.0-AWQ | LlamaForCausalLM | ✅ | — | | TheBloke/TinyLlama-1.1B-Chat-v1.0-GPTQ | LlamaForCausalLM | ✅ | — | -| ibm-granite/granite-3.2-2b-instruct | GraniteForCausalLM | ✅ | — | +| ibm-granite/granite-3.2-2b-instruct | GraniteForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/ibm-granite/granite-3.2-2b-instruct?hardware=xeon6) | | Qwen/Qwen3-1.7B | Qwen3ForCausalLM | ✅ | — | | Qwen/Qwen3-4B | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-4B?hardware=xeon6) | | Qwen/Qwen3-8B | Qwen3ForCausalLM | ✅ | — | -| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | — | -| Qwen/Qwen3-14B-FP8 | Qwen3ForCausalLM | ✅ | — | -| Qwen/Qwen3-14B-AWQ | Qwen3ForCausalLM | ✅ | — | +| Qwen/Qwen3-14B | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-14B?hardware=xeon6) | +| Qwen/Qwen3-14B-FP8 | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-14B?hardware=xeon6&variant=fp8) | +| Qwen/Qwen3-14B-AWQ | Qwen3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-14B?hardware=xeon6&variant=awq) | | Qwen/Qwen3-30B-A3B | Qwen3MoeForCausalLM | ✅ | — | | Qwen/Qwen3-30B-A3B-Instruct-2507-FP8 | Qwen3MoeForCausalLM | ✅ | — | -| Qwen/QwQ-32B | Qwen2ForCausalLM | ✅ | — | -| Qwen/QwQ-32B-AWQ | Qwen2ForCausalLM | ✅ | — | +| Qwen/QwQ-32B | Qwen2ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/QwQ-32B?hardware=xeon6) | +| Qwen/QwQ-32B-AWQ | Qwen2ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/QwQ-32B?hardware=xeon6&variant=awq) | | Qwen/Qwen1.5-0.5B-Chat-GPTQ-Int4 | Qwen2ForCausalLM | ✅ | — | -| RedHatAI/QwQ-32B-quantized.w8a8 | Qwen2ForCausalLM | ✅ | — | -| zai-org/glm-4-9b-hf | GLMForCausalLM | ✅ | — | +| RedHatAI/QwQ-32B-quantized.w8a8 | Qwen2ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/QwQ-32B?hardware=xeon6&variant=w8a8_int8) | +| zai-org/glm-4-9b-hf | GLMForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/zai-org/glm-4-9b-hf?hardware=xeon6) | | google/gemma-7b | GemmaForCausalLM | ✅ | — | -| microsoft/Phi-4-reasoning | Phi3ForCausalLM | ✅ | — | +| microsoft/Phi-4-reasoning | Phi3ForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/microsoft/Phi-4-reasoning?hardware=xeon6) | | mistralai/Mistral-7B-Instruct-v0.2 | MistralForCausalLM | ✅ | — | | TheBloke/Mistral-7B-Instruct-v0.2-AWQ | MistralForCausalLM | ✅ | — | @@ -75,9 +76,9 @@ vllm serve --config config.yaml | google/gemma-4-E4B-it | Gemma4ForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Google/gemma-4-E4B-it?hardware=xeon6) | | google/gemma-4-E2B-it | Gemma4ForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Google/gemma-4-E2B-it?hardware=xeon6) | | google/gemma-4-26B-A4B-it | Gemma4ForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Google/gemma-4-26B-A4B-it?hardware=xeon6) | -| microsoft/Phi-4-multimodal-instruct | Phi4MMForCausalLM | ✅ | — | -| Qwen/Qwen2.5-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ | — | -| Qwen/Qwen3-VL-30B-A3B-Instruct | Qwen3VLMoeForConditionalGeneration | ✅ | — | -| openai/whisper-large-v3 | WhisperForConditionalGeneration | ✅ | — | +| microsoft/Phi-4-multimodal-instruct | Phi4MMForCausalLM | ✅ | [Xeon 6](https://recipes.vllm.ai/microsoft/Phi-4-multimodal-instruct?hardware=xeon6) | +| Qwen/Qwen2.5-VL-7B-Instruct | Qwen2VLForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen2.5-VL-7B-Instruct?hardware=xeon6) | +| Qwen/Qwen3-VL-30B-A3B-Instruct | Qwen3VLMoeForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/Qwen/Qwen3-VL-30B-A3B-Instruct?hardware=xeon6) | +| openai/whisper-large-v3 | WhisperForConditionalGeneration | ✅ | [Xeon 6](https://recipes.vllm.ai/openai/whisper-large-v3?hardware=xeon6) | ✅ Runs and optimized.