vLLM/recipes/models/LiquidAI/LFM2.5-VL-1.6B.yaml

meta:
  title: "LFM2.5 VL 1.6B"
  slug: "lfm2.5-vl-1.6b"
  provider: "LiquidAI"
  description: "Liquid AI's 1.6B vision-language model — LFM2 hybrid LM backbone plus a SigLIP2 vision tower for image+text chat on a single small GPU."
  date_added: 2026-06-25
  date_updated: 2026-06-24
  tasks:
    - multimodal
  performance_headline: "1.6B vision-language model (hybrid LM + SigLIP2 vision) — image understanding on a single small GPU"
  related_recipes:
    - LiquidAI/LFM2.5-VL-450M
    - LiquidAI/LFM2.5-1.2B-Instruct
  hardware:
    h100: verified
    h200: verified
    b200: verified

model:
  model_id: "LiquidAI/LFM2.5-VL-1.6B"
  min_vllm_version: "0.23.0"
  architecture: dense
  parameter_count: "1.6B"
  active_parameters: "1.6B"
  context_length: 32768
  base_args: []
  base_env: {}
  # TODO(vincentzed): the Lfm2VL encoder CUDA graph (vllm-project/vllm#44930, ~10-20% lower e2e
  # latency at low batch) is NOT in 0.23.0 — add the encoder-CG compilation flag here once it
  # ships in a stable release (>0.23.0).

variants:
  default:
    precision: bf16
    vram_minimum_gb: 4
    description: "Full BF16 (LM + vision encoder) — single 8 GB+ NVIDIA GPU"

compatible_strategies:
  - single_node_tp

hardware_overrides:
  # Blackwell (B200) only: route RMSNorm to the bundled oink kernels (measured ~+2.6% on B200,
  # output identical). Inert (falls back to native) on non-Blackwell GPUs.
  blackwell:
    extra_env:
      VLLM_USE_OINK_OPS: "1"
strategy_overrides:
  single_node_tp:
    tp: 1

guide: |
  ## Overview

  [LFM2.5-VL-1.6B](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B) is the larger vision-language
  model in [Liquid AI's](https://www.liquid.ai/) LFM2.5-VL family. It pairs the **LFM2 hybrid
  language backbone** (short-range gated convolution blocks interleaved with grouped-query
  attention) with a SigLIP2 vision encoder for image understanding, while remaining small enough
  to serve on a single commodity GPU through vLLM's OpenAI-compatible API.

  ### Key Features
  - **Vision-language**: SigLIP2 vision encoder on top of the LFM2 hybrid language backbone — single- and multi-image prompts.
  - **Hybrid LM backbone**: Gated short convolutions interleaved with grouped-query attention — a smaller KV cache and lower decode latency than a same-size full-attention transformer.
  - **32K context**: 32,768-token context window.
  - **Native vLLM support**: Served via the `Lfm2VlForConditionalGeneration` architecture.

  ### Supported Variants

  Vision-Language:
  - `LiquidAI/LFM2.5-VL-450M` (450M)
  - `LiquidAI/LFM2.5-VL-1.6B` (1.6B)

  Text (same LFM2 family):
  - Dense: `LiquidAI/LFM2.5-230M`, `LiquidAI/LFM2.5-350M`, `LiquidAI/LFM2.5-1.2B-Instruct`, `LiquidAI/LFM2.5-1.2B-Thinking`, `LiquidAI/LFM2.5-1.2B-JP`, `LiquidAI/LFM2.5-1.2B-JP-202606`, `LiquidAI/LFM2.5-1.2B-Base`
  - MoE: `LiquidAI/LFM2.5-8B-A1B`

  See the [LFM2.5 usage guide](../../LiquidAI/LFM2.5) for the full family.

  ## Prerequisites

  - **Hardware**: 1× GPU with ≥8 GB VRAM.
  - **vLLM**: ≥ 0.23.0.

  ### pip (NVIDIA CUDA)
  ```bash
  uv venv
  source .venv/bin/activate
  uv pip install -U vllm --torch-backend auto
  ```

  ## Deployment Configurations

  ### Quick Start (Single GPU, BF16)
  ```bash
  vllm serve LiquidAI/LFM2.5-VL-1.6B
  ```

  ### Multiple Images per Request
  ```bash
  vllm serve LiquidAI/LFM2.5-VL-1.6B \
    --limit-mm-per-prompt '{"image": 4}' \
    --host 0.0.0.0 --port 8000
  ```

  ### Docker (NVIDIA)
  ```bash
  docker run -itd --name lfm2.5-vl-1.6b \
    --ipc=host --network host --shm-size 16G --gpus all \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    vllm/vllm-openai:latest \
      --model LiquidAI/LFM2.5-VL-1.6B \
      --host 0.0.0.0 --port 8000
  ```

  ## Client Usage

  ### Image Understanding
  Send an image + text turn via the OpenAI chat API. The card recommends temperature `0.1`,
  `min_p 0.15`, `repetition_penalty 1.05` (`min_p` and `repetition_penalty` ride in `extra_body`).
  ```python
  from openai import OpenAI
  client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
  response = client.chat.completions.create(
      model="LiquidAI/LFM2.5-VL-1.6B",
      messages=[{
          "role": "user",
          "content": [
              {"type": "image_url", "image_url": {"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"}},
              {"type": "text", "text": "What is in this image?"},
          ],
      }],
      temperature=0.1,
      extra_body={"min_p": 0.15, "repetition_penalty": 1.05},
  )
  print(response.choices[0].message.content)
  ```

  ### Multiple Images
  Launch with `--limit-mm-per-prompt '{"image": N}'`, then include several `image_url` blocks in
  one message to compare or reason across images.

  ## Performance

  Tensor parallelism gives no speedup — the model fits on a single GPU.

  ## Configuration Tips
  - `--limit-mm-per-prompt '{"image": N}'` caps images per request (default 1).
  - Set `--max-model-len` to match your workload (up to 32K).
  - `--gpu-memory-utilization 0.90–0.95` maximizes KV cache capacity.
  - Sampling presets are per-request client defaults — don't bake them into `vllm serve`.

  ## References
  - [Model card](https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B)
  - [LFM2.5 usage guide](../../LiquidAI/LFM2.5)
  - [Liquid AI](https://www.liquid.ai/)
  - [vLLM supported models](https://docs.vllm.ai/en/latest/models/supported_models.html)