Roy WangandClaude Opus 5.5
b515648641
Signed-off-by: esmeetu <jasonailu87@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
138 lines
3.2 KiB
YAML
138 lines
3.2 KiB
YAML
meta:
|
|
title: "MiMo-V2.5-Pro"
|
|
slug: "mimo-v2-5-pro"
|
|
provider: "MiMo (Xiaomi)"
|
|
description: "Xiaomi's flagship MoE reasoning model (1.02T total / 42B active) with hybrid attention, native FP8 weights, and Multi-Token Prediction"
|
|
date_added: 2026-04-27
|
|
date_updated: 2026-04-27
|
|
tasks:
|
|
- text
|
|
related_recipes:
|
|
- "XiaomiMiMo/MiMo-V2.5"
|
|
hardware:
|
|
h200: verified
|
|
|
|
model:
|
|
model_id: "XiaomiMiMo/MiMo-V2.5-Pro"
|
|
min_vllm_version: "0.21.0"
|
|
architecture: moe
|
|
parameter_count: "1T"
|
|
active_parameters: "42B"
|
|
context_length: 1048576
|
|
base_args:
|
|
- "--trust-remote-code"
|
|
- "--max-model-len auto"
|
|
- "--generation-config"
|
|
- "vllm"
|
|
base_env: {}
|
|
|
|
features:
|
|
tool_calling:
|
|
description: "MiMo tool-call parser"
|
|
args:
|
|
- "--tool-call-parser"
|
|
- "mimo"
|
|
- "--enable-auto-tool-choice"
|
|
reasoning:
|
|
description: "MiMo reasoning parser"
|
|
args:
|
|
- "--reasoning-parser"
|
|
- "mimo"
|
|
spec_decoding:
|
|
description: "Multi-Token Prediction"
|
|
args:
|
|
- "--speculative-config"
|
|
- '{"method":"mtp","num_speculative_tokens":1}'
|
|
|
|
opt_in_features:
|
|
- spec_decoding
|
|
|
|
variants:
|
|
default:
|
|
precision: fp8
|
|
vram_minimum_gb: 1224
|
|
description: "Native FP8 weights (block-wise e4m3 128x128); 8x H200 with TP8"
|
|
extra_args:
|
|
- "--tensor-parallel-size"
|
|
- "8"
|
|
- "--gpu-memory-utilization"
|
|
- "0.95"
|
|
- "--max-model-len"
|
|
- "auto"
|
|
|
|
compatible_strategies:
|
|
- single_node_tp
|
|
- single_node_tep
|
|
- pd_cluster
|
|
|
|
hardware_overrides: {}
|
|
strategy_overrides: {}
|
|
|
|
guide: |
|
|
## Overview
|
|
|
|
MiMo-V2.5-Pro is Xiaomi's flagship MoE reasoning model with 1.02T total parameters
|
|
and 42B active per token. It uses 384 routed experts (top-8) with hybrid
|
|
attention (full-attention + sliding-window 128 at 6:1 ratio) over 70 layers
|
|
(1 dense + 69 MoE) and ships with native FP8 (block-wise e4m3) weights.
|
|
A 3-layer Multi-Token Prediction (MTP) head enables speculative decoding for
|
|
~3x output speed.
|
|
|
|
## Prerequisites
|
|
|
|
- Hardware: 8x H200 (TP8)
|
|
|
|
### Pull the vLLM docker image
|
|
|
|
Stable vLLM does not yet support MiMo V2.5. Use the pre-built image:
|
|
|
|
```bash
|
|
docker pull vllm/vllm-openai:mimov25-cu129
|
|
```
|
|
|
|
## Launch commands
|
|
|
|
Single-node TP8 (H200):
|
|
|
|
```bash
|
|
vllm serve XiaomiMiMo/MiMo-V2.5-Pro \
|
|
--tensor-parallel-size 8 \
|
|
--trust-remote-code \
|
|
--gpu-memory-utilization 0.95 \
|
|
--max-model-len auto \
|
|
--generation-config vllm
|
|
```
|
|
|
|
With tool calling + reasoning:
|
|
|
|
```bash
|
|
vllm serve XiaomiMiMo/MiMo-V2.5-Pro \
|
|
--tensor-parallel-size 8 \
|
|
--trust-remote-code \
|
|
--gpu-memory-utilization 0.95 \
|
|
--max-model-len auto \
|
|
--reasoning-parser mimo \
|
|
--tool-call-parser mimo \
|
|
--enable-auto-tool-choice \
|
|
--generation-config vllm
|
|
```
|
|
|
|
## Client Usage
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "XiaomiMiMo/MiMo-V2.5-Pro",
|
|
"messages": [{"role": "user", "content": "Hello MiMo!"}],
|
|
"chat_template_kwargs": {"enable_thinking": true}
|
|
}'
|
|
```
|
|
|
|
Set `"enable_thinking": false` (or omit the kwargs) to disable thinking mode.
|
|
|
|
|
|
## References
|
|
|
|
- [MiMo-V2.5-Pro on Hugging Face](https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Pro)
|