vLLM/recipes/models/LiquidAI/LFM2.5-1.2B-JP.yaml

meta:
  title: "LFM2.5 1.2B JP"
  slug: "lfm2.5-1.2b-jp"
  provider: "LiquidAI"
  description: "Liquid AI's 1.2B Japanese-specialized chat model on the LFM2 hybrid conv+attention backbone, with a 32K context window on a single small GPU."
  date_added: 2026-06-25
  date_updated: 2026-06-24
  tasks:
    - text
  performance_headline: "Japanese-tuned 1.2B hybrid chat model — single small GPU"
  related_recipes:
    - LiquidAI/LFM2.5-1.2B-JP-202606
    - LiquidAI/LFM2.5-1.2B-Instruct
  hardware:
    h100: verified
    h200: verified
    b200: verified

model:
  model_id: "LiquidAI/LFM2.5-1.2B-JP"
  min_vllm_version: "0.23.0"
  architecture: dense
  parameter_count: "1.2B"
  active_parameters: "1.2B"
  context_length: 32768
  base_args: []
  base_env: {}

variants:
  default:
    precision: bf16
    vram_minimum_gb: 3
    description: "Full BF16 — single 8 GB+ NVIDIA GPU with room for KV cache"

compatible_strategies:
  - single_node_tp

hardware_overrides:
  # Blackwell (B200) only: route RMSNorm to the bundled oink kernels (measured ~+2.6% on B200,
  # output identical). Inert (falls back to native) on non-Blackwell GPUs.
  blackwell:
    extra_env:
      VLLM_USE_OINK_OPS: "1"
strategy_overrides:
  single_node_tp:
    tp: 1

guide: |
  ## Overview

  [LFM2.5-1.2B-JP](https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP) is the Japanese-specialized
  1.2B chat model from [Liquid AI](https://www.liquid.ai/), built on the **LFM2 hybrid backbone**
  (short-range gated convolution blocks interleaved with grouped-query attention). It serves on a
  single small GPU through vLLM's OpenAI-compatible API. For the latest Japanese checkpoint with
  tool calling, see
  [LFM2.5-1.2B-JP-202606](https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP-202606).

  ### Key Features
  - **Japanese-specialized**: Tuned for Japanese chat and instruction following.
  - **Hybrid backbone**: Gated short convolutions interleaved with grouped-query attention — a smaller KV cache and lower decode latency than a same-size full-attention transformer.
  - **32K context**: 32,768-token context window.
  - **Native vLLM support**: Served via the `Lfm2ForCausalLM` architecture.

  ### Supported Variants

  Dense:
  - `LiquidAI/LFM2.5-230M` (230M)
  - `LiquidAI/LFM2.5-350M` (350M)
  - `LiquidAI/LFM2.5-1.2B-Instruct` (1.2B)
  - `LiquidAI/LFM2.5-1.2B-Thinking` (1.2B, reasoning)
  - `LiquidAI/LFM2.5-1.2B-JP` / `LiquidAI/LFM2.5-1.2B-JP-202606` (Japanese)
  - `LiquidAI/LFM2.5-1.2B-Base` (pretrained base)

  MoE:
  - `LiquidAI/LFM2.5-8B-A1B` (8B total / ~1B active)

  Vision-Language:
  - `LiquidAI/LFM2.5-VL-450M`, `LiquidAI/LFM2.5-VL-1.6B`

  See the [LFM2.5 usage guide](../../LiquidAI/LFM2.5) for the full family.

  ## Prerequisites

  - **Hardware**: 1× GPU with ≥8 GB VRAM.
  - **vLLM**: ≥ 0.23.0.

  ### pip (NVIDIA CUDA)
  ```bash
  uv venv
  source .venv/bin/activate
  uv pip install -U vllm --torch-backend auto
  ```

  ## Deployment Configurations

  ### Quick Start (Single GPU, BF16)
  ```bash
  vllm serve LiquidAI/LFM2.5-1.2B-JP
  ```

  ### Docker (NVIDIA)
  ```bash
  docker run -itd --name lfm2.5-jp \
    --ipc=host --network host --shm-size 16G --gpus all \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    vllm/vllm-openai:latest \
      --model LiquidAI/LFM2.5-1.2B-JP \
      --host 0.0.0.0 --port 8000
  ```

  ## Client Usage

  ### Text Generation
  The card recommends temperature `0.3`, `min_p 0.15`, `repetition_penalty 1.05` for the JP
  checkpoint (`min_p` and `repetition_penalty` ride in `extra_body`).
  ```python
  from openai import OpenAI
  client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
  response = client.chat.completions.create(
      model="LiquidAI/LFM2.5-1.2B-JP",
      messages=[{"role": "user", "content": "日本の四季について簡単に説明してください。"}],
      temperature=0.3,
      extra_body={"min_p": 0.15, "repetition_penalty": 1.05},
  )
  print(response.choices[0].message.content)
  ```

  ## Configuration Tips
  - Set `--max-model-len` to match your workload (up to 32K); lowering it frees VRAM for KV cache.
  - `--gpu-memory-utilization 0.90–0.95` maximizes KV cache capacity.
  - Sampling presets are per-request client defaults — don't bake them into `vllm serve`.

  ## References
  - [Model card](https://huggingface.co/LiquidAI/LFM2.5-1.2B-JP)
  - [LFM2.5 usage guide](../../LiquidAI/LFM2.5)
  - [Liquid AI](https://www.liquid.ai/)
  - [vLLM supported models](https://docs.vllm.ai/en/latest/models/supported_models.html)