[Doc] Update model support information (#52289)
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
This commit is contained in:
@@ -572,6 +572,7 @@ These models primarily accept the [`LLM.generate`](./generative_models.md#llmgen
|
||||
| `MossAudioModel` | MOSS-Audio | T + A<sup>+</sup> | `OpenMOSS-Team/MOSS-Audio-4B-Instruct`, `OpenMOSS-Team/MOSS-Audio-4B-Thinking`, `OpenMOSS-Team/MOSS-Audio-8B-Instruct`, `OpenMOSS-Team/MOSS-Audio-8B-Thinking` | ✅︎ | ✅︎ |
|
||||
| `MossTranscribeDiarizeForConditionalGeneration` | MOSS-Transcribe-Diarize | T + A | `OpenMOSS-Team/MOSS-Transcribe-Diarize` | | ✅︎ |
|
||||
| `Moondream3ForCausalLM` | Moondream3 | T + I | `moondream/moondream3-preview` | | ✅︎ |
|
||||
| `MuseGlimmerForCausalLM`, `MuseGlimmerForConditionalGeneration` | Muse Glimmer | T + I<sup>+</sup> + V<sup>+</sup> | `meta-models/Muse-Glimmer-30B` | ✅︎ | ✅︎ |
|
||||
| `NVLM_D_Model` | NVLM-D 1.0 | T + I<sup>+</sup> | `nvidia/NVLM-D-72B`, etc. | | ✅︎ |
|
||||
| `OpenCUAForConditionalGeneration` | OpenCUA-7B | T + I<sup>E+</sup> | `xlangai/OpenCUA-7B` | ✅︎ | ✅︎ |
|
||||
| `OpenPanguVLForConditionalGeneration` | openpangu-VL | T + I<sup>E+</sup> + V<sup>E+</sup> | `FreedomIntelligence/openPangu-VL-7B` | ✅︎ | ✅︎ |
|
||||
@@ -619,6 +620,13 @@ Some models are supported only via the [Transformers modeling backend](#transfor
|
||||
<sup>*</sup> Only specific variants of the model support this modality (see notes below).</br>
|
||||
<sup>Q</sup> `Qwen*-VL` officially uses `qwen_vl_utils` for image preprocessing, while vLLM uses `transformers`' `video_processing_qwen*`, which leads to slightly different results compared to the official Hugging Face repository examples.
|
||||
|
||||
!!! note
|
||||
For `Dots3NoteForCausalLM`, the vision and audio towers are only loaded when the
|
||||
corresponding modality is enabled via `--limit-mm-per-prompt`. Video inputs are
|
||||
decoded into frames and audio, so they require both towers. The checkpoint also
|
||||
ships one MTP layer, enabled with
|
||||
`--speculative-config '{"method":"mtp","num_speculative_tokens":1}'`.
|
||||
|
||||
!!! note
|
||||
`Gemma3nForConditionalGeneration` is only supported on V1 due to shared KV caching and it depends on `timm>=1.0.17` to make use of its
|
||||
MobileNet-v5 vision backbone.
|
||||
@@ -656,6 +664,13 @@ Some models are supported only via the [Transformers modeling backend](#transfor
|
||||
coordinate decoding and are not exposed by this vLLM implementation.
|
||||
See [Moondream3 prompt recipes](../features/multimodal_inputs.md#moondream3-prompt-recipes).
|
||||
|
||||
!!! note
|
||||
Both Muse Glimmer architecture names map to the same vLLM implementation:
|
||||
checkpoints with a vision config accept image and video inputs, while
|
||||
vision-less checkpoints run as a text-only model. Speculative decoding uses the
|
||||
`meta-models/Muse-Glimmer-30B-assistant` checkpoint, which vLLM serves through
|
||||
its [DFlash](../features/speculative_decoding/README.md) path.
|
||||
|
||||
!!! note
|
||||
The official `openbmb/MiniCPM-V-2` doesn't work yet, so we need to use a fork (`HwwwH/MiniCPM-V-2`) for now.
|
||||
For more details, please see: <https://github.com/vllm-project/vllm/pull/4087#issuecomment-2250397630>
|
||||
|
||||
@@ -847,8 +847,7 @@ _MULTIMODAL_EXAMPLE_MODELS = {
|
||||
"deepseek-ai/DeepSeek-OCR-2",
|
||||
),
|
||||
"Dots3NoteForCausalLM": _HfExamplesInfo(
|
||||
"rednote-hilab/dots3.note",
|
||||
trust_remote_code=True,
|
||||
"dots-studio/dots3-note-prev",
|
||||
is_available_online=False,
|
||||
),
|
||||
"UnlimitedOCRForCausalLM": _HfExamplesInfo(
|
||||
@@ -1681,9 +1680,8 @@ _SPECULATIVE_DECODING_EXAMPLE_MODELS = {
|
||||
is_available_online=False,
|
||||
),
|
||||
"Dots3NoteMTPModel": _HfExamplesInfo(
|
||||
"rednote-hilab/dots3.note",
|
||||
speculative_model="rednote-hilab/dots3.note",
|
||||
trust_remote_code=True,
|
||||
"dots-studio/dots3-note-prev",
|
||||
speculative_model="dots-studio/dots3-note-prev",
|
||||
is_available_online=False,
|
||||
),
|
||||
"Gemma4MTPModel": _HfExamplesInfo(
|
||||
|
||||
Reference in New Issue
Block a user