MOVZXandHinata 6cf86cd95f [Bugfix] Load checkpoint KV-cache scales in the Qwen3.5 family ...

Checkpoints that ship calibrated k_scale/v_scale tensors (e.g.
unsloth/Qwen3.8-27B-NVFP4, which carries one pair per full-attention
layer) name them ...self_attn.{k,v}_scale, but the runtime parameters
live on the inner attention layer (...self_attn.attn.{k,v}_scale).
Qwen3_5Model.load_weights hands the iterable straight to
AutoWeightsLoader, which performs no scale-name remapping, so the
scales are silently dropped and --kv-cache-dtype fp8 runs at scale 1.0
with only a warning_once that fires even on success (the MTP draft
layer legitimately has no scales).

Route scale-suffixed names through maybe_remap_kv_scale_name first,
mirroring gemma4/gpt_oss. No behavior change for checkpoints without
scales.

Measured on RTX 5090 / Qwen3.8-27B NVFP4 / fp8 KV / MTP k=3 (GSM8K
greedy, n=300): 92.3% at scale 1.0 vs 93.7% with scales loaded
(bf16-KV control 94.5%).

Fixes #54623

Co-Authored-By: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>

2026-10-18 17:18:47 +00:00

..

transformers

[Bugfix][Transformers backend] Fix CUDA graph capture for dynamic-rope models (#58064)

2026-10-14 05:56:54 -07:00

__init__.py

[Qwen4Exp] support Sequence Parallelism for Qwen3.8-Flash-Next (#56322)

2026-10-12 20:56:44 +08:00

abliteration.py

fix: keep the refusal-direction projection dynamo-safe

2026-10-15 22:56:06 +07:00

adapters.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

afmoe.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

aimv2.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

alice_ai.py

[Model] Add AliceAI support (#57963)

2026-10-15 10:15:03 +00:00

apertus.py

[Model] Remove ten deprecated model architectures (#53608)

2026-08-24 17:06:00 -07:00

aria.py

fix: return loaded parameters in Aria load_weights (#57476)

2026-09-29 04:05:07 +00:00

audioflamingo3.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

AXK1.py

[Bugfix][Model] Restore AXK1 config defaults dropped with the vendored config (#60954)

2026-10-10 05:54:52 +00:00

bagel.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

bailing_moe_linear.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

bailing_moe_mtp.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

bailing_moe_v3_mtp.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

bailing_moe_v3_vl.py

[LoRA] Code cleanup (#60017)

2026-10-05 09:15:53 +00:00

bailing_moe_v3.py

[Bugfix] Fix TP loading of small block-FP8 MLA gates in BailingMoeV3 (#61642)

2026-10-14 10:09:23 -07:00

bailing_moe.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

bee.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

bert_with_rope.py

[Model] Migrate Glm, Arcee, CWM and Mellum to the Transformers modeling backend (#59679)

2026-10-02 03:15:14 +00:00

bert.py

[Model][LoRA] Enable LoRA support for BERT and RoBERTa embedding models (#61033)

2026-10-12 14:09:53 +00:00

blip2.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

blip.py

[Refactor] Remove dead code multiple places (#60120)

2026-10-10 15:41:10 +00:00

bloom.py

Fix weight tying (#51665)

2026-08-20 10:31:55 +00:00

chatglm.py

[Mypy] Fix mypy typing for model interfaces and H/I models (#54079)

2026-08-28 16:49:19 -04:00

clef_flash.py

[Model] Add Clef-Flash support (#61308)

2026-10-13 05:46:37 +00:00

clef_joint_schema.py

[Model] Add Clef-Flash support (#61308)

2026-10-13 05:46:37 +00:00

clef_omni.py

[Model] Add Clef-Flash support (#61308)

2026-10-13 05:46:37 +00:00

clip.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

cohere2_moe.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

cohere2_vision.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

cohere_asr.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

cohere_compass.py

[LoRA] Code cleanup (#60017)

2026-10-05 09:15:53 +00:00

cohere_eagle.py

Reduce AutoWeightsLoader kwargs (#53106)

2026-08-20 15:40:11 +00:00

colbert.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

colmodernvbert.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

colpali.py

[MM] Further cleanup _apply_hf_processor_main (#53610)

2026-09-16 10:21:00 -07:00

colqwen3_5.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

colqwen3.py

[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next (#51289)

2026-09-29 11:44:54 +08:00

commandr.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

config.py

[ROCm][Qwen4Exp] Support HC and MoE sequence parallelism on ROCm (#61471)

2026-10-15 20:11:17 -05:00

conformer_encoder.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

cosmos3_edge.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

cosmos3.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

d3.py

[Model] Support every Decision 3.0 size (d3-flash, d3-mini, d3-nano, d3-lite, d3-edge) (#61778)

2026-10-15 13:29:24 +00:00

dbrx.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

deepencoder2.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

deepencoder.py

[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)

2026-09-16 10:57:24 -07:00

deepseek_eagle3.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

deepseek_eagle.py

Reduce AutoWeightsLoader kwargs (#53106)

2026-08-20 15:40:11 +00:00

deepseek_mtp.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

deepseek_ocr2.py

[Security] Accept zero-sum DeepSeek-OCR pixel tensors (#59417)

2026-10-02 09:17:12 +00:00

deepseek_ocr.py

[Security] Accept zero-sum DeepSeek-OCR pixel tensors (#59417)

2026-10-02 09:17:12 +00:00

deepseek_v2.py

[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686)

2026-10-08 12:04:46 -04:00

deepseek_vl2.py

[MM][CG] Support ViT full CUDA graph for DeepSeek-VL2 (#46005)

2026-10-15 21:38:52 +08:00

diffusion_gemma_sampler.py

[Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226)

2026-09-25 14:14:55 -04:00

diffusion_gemma.py

[Perf][Sampler] Speed up the small-batch split top-p pipeline (#61996)

2026-10-16 21:22:11 -07:00

dots_ocr.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

eagle2_5_vl.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

embedding_gemma2.py

[Model] Enable torch.compile and CUDA graphs for EmbeddingGemma2 (#61572)

2026-10-15 12:32:39 +08:00

ernie45_moe.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

ernie45_vl_moe.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

ernie45_vl.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

ernie45.py

…

ernie_mtp.py

Fix weight tying (#51665)

2026-08-20 10:31:55 +00:00

exaone4_5_mtp.py

[Bugfix][Model] Fix EXAONE-4.5 image inputs: ViT received un-normalised pixels (#61104)

2026-10-13 13:02:12 +08:00

exaone4_5.py

[Bugfix][Model] Fix EXAONE-4.5 image inputs: ViT received un-normalised pixels (#61104)

2026-10-13 13:02:12 +08:00

exaone4.py

Fix weight tying (#51665)

2026-08-20 10:31:55 +00:00

exaone_moe_mtp.py

[Mypy Fix] Mypy fix for "vllm/model_executor/models/[eE][fF]" (#53381)

2026-08-22 15:41:27 -04:00

exaone_moe.py

[Bugfix][Model] K-EXAONE: restore loading of pre-2.0 configs and apply RoPE to the MTP layer (#60187)

2026-10-10 15:05:55 +00:00

exaone.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

expert_substitution.py

Add MoNe experts pruning support (#50415)

2026-10-12 23:18:43 +00:00

extract_hidden_states.py

[Attention] Remove cascade attention (#61126)

2026-10-12 10:41:07 -07:00

falcon_h1.py

[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)

2026-10-13 00:58:59 -07:00

falcon.py

[Mypy Fix] Mypy fix for "vllm/model_executor/models/[eE][fF]" (#53381)

2026-08-22 15:41:27 -04:00

fireredasr2.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

funasr.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

funaudiochat.py

[Bugfix] Use the correct repository revision for secondary artifact loaders (#57461)

2026-09-28 14:25:40 +00:00

gemma2.py

Fix weight tying (#51665)

2026-08-20 10:31:55 +00:00

gemma3_mm.py

[ROCM][Bugfix] Deferring to TRITON_ATTN backend for Prefix lm models instead of ROCM_ATTN and fixed minor bug in Gemma3 Multimodal code (#46754)

2026-10-13 10:03:48 -05:00

gemma3.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

gemma3n_audio_utils.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

gemma3n_mm.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

gemma3n.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

gemma4_dspark.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

gemma4_mm.py

[Bugfix] Fix Gemma 4 encoder CUDA graph capture OOM at startup (#61365)

2026-10-17 11:04:49 +00:00

gemma4_mtp.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

gemma4_unified.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

gemma4.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

gemma.py

[Mypy] Fix mypy typing for model interfaces and H/I models (#54079)

2026-08-28 16:49:19 -04:00

glm4_1v.py

[Model] Extend device-side mm normalization to GLM4V/GLM5Next (#55389)

2026-09-30 15:44:46 +08:00

glm4_moe_lite_mtp.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

glm4_moe_lite.py

[3/N] HiSparse: host-resident sparse-MLA decode hot-buffering (#53781)

2026-09-12 00:36:47 -04:00

glm4_moe_mtp.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

glm4_moe.py

[Quantization] Enable shared expert fusion compatibility with online shared_expert quantization (showcase: along Quark MXFP4 routed experts) (#55686)

2026-10-08 12:04:46 -04:00

glm4.py

Reduce AutoWeightsLoader kwargs (#53106)

2026-08-20 15:40:11 +00:00

glm4v.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

glm_ocr_mtp.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

glm_ocr.py

[Model] Extend device-side mm normalization to GLM4V/GLM5Next (#55389)

2026-09-30 15:44:46 +08:00

glmasr_utils.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

glmasr.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

gpt2.py

[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172)

2026-09-30 12:19:33 +00:00

gpt_j.py

[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172)

2026-09-30 12:19:33 +00:00

gpt_oss.py

[Model Runner V2] Enable batch-sharded sampling for GPT-OSS (#61257)

2026-10-14 10:11:11 -07:00

granite4_vision.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

granite_speech_plus.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

granite_speech.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

granite.py

[Model] Support EAGLE3/DFlash aux hidden states for Granite (#60855)

2026-10-10 13:34:10 +08:00

granitemoe.py

[Model] Support EAGLE3/DFlash aux hidden states for GraniteMoe and GraniteMoeShared (#61110)

2026-10-14 12:01:14 -04:00

granitemoehybrid.py

[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)

2026-10-13 00:58:59 -07:00

granitemoeshared.py

[Model] Support EAGLE3/DFlash aux hidden states for GraniteMoe and GraniteMoeShared (#61110)

2026-10-14 12:01:14 -04:00

h2ovl.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

hrm_text.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

hy_v3_mtp.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

hy_v3.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

hyperclovax_vision_v2.py

[MM] Move get_dummy_processor_inputs into MM processor (#57967)

2026-09-21 09:39:20 -07:00

hyperclovax.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

idefics2_vision_model.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

idefics3.py

[Bugfix] Count unsplit Idefics3 image patches (#48760)

2026-09-24 11:36:26 -07:00

interfaces_base.py

[Feature][Model Loader] Apply model-owned MTP shard selection to RunAI ModelStreamer (#60759)

2026-10-15 12:23:11 +08:00

interfaces.py

Merge branch 'main' into patches

2026-10-17 08:06:56 +00:00

intern_vit.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

internlm2.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

interns1_pro.py

[Model] Extend device-side mm normalization to Qwen3VL/Qwen3.5/Qwen4Next (#51289)

2026-09-29 11:44:54 +08:00

interns1_vit.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

interns1.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

interns2_mobius.py

[Core] Remove deprecated mamba_cache_mode "all" (#58997)

2026-10-01 02:59:37 -07:00

interns2_preview.py

Reduce AutoWeightsLoader kwargs (#53106)

2026-08-20 15:40:11 +00:00

internvl.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

iquest_loopcoder.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

isaac.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

jamba.py

[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)

2026-10-13 00:58:59 -07:00

jina_vl.py

[MM] Further cleanup _apply_hf_processor_main (#53610)

2026-09-16 10:21:00 -07:00

jina.py

[Mypy] Fix typing for J models (#54130)

2026-08-29 08:50:15 -04:00

k2_horizon.py

[MyPy] Fix mypy errors in vllm/model_executor/models/[kK]* (#59402)

2026-10-01 19:32:27 +00:00

kanana_v.py

[MyPy] Fix mypy errors in vllm/model_executor/models/[kK]* (#59402)

2026-10-01 19:32:27 +00:00

keye_vl1_5.py

[MyPy] Fix mypy errors in vllm/model_executor/models/[kK]* (#59402)

2026-10-01 19:32:27 +00:00

keye.py

[Refactor] Remove dead code multiple places (#60120)

2026-10-10 15:41:10 +00:00

kimi_audio.py

[MyPy] Fix mypy errors in vllm/model_executor/models/[kK]* (#59402)

2026-10-01 19:32:27 +00:00

kimi_k25_vit.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

kimi_k25.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

kimi_vl.py

[MyPy] Fix mypy errors in vllm/model_executor/models/[kK]* (#59402)

2026-10-01 19:32:27 +00:00

laguna_dflash.py

[Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514)

2026-09-05 11:45:34 +08:00

laguna.py

[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797)

2026-10-14 10:17:58 +00:00

lfm2_moe.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

lfm2_siglip2.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

lfm2_vl.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

lfm2.py

[Core] Remove deprecated mamba_cache_mode "all" (#58997)

2026-10-01 02:59:37 -07:00

lightonocr.py

[Model] Support precomputed image embeddings for LightOnOCR (#61585)

2026-10-14 10:54:09 +00:00

lilicorr.py

[SpecDecode] Add LiLiCorr drafter (#57934)

2026-09-25 10:08:28 -04:00

llama4_eagle.py

[Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters (#57633)

2026-10-09 01:44:50 +00:00

llama4.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

llama_eagle3.py

[Bugfix][Spec Decode] Only create draft_id_to_target_id when draft vocab differs (#53458)

2026-09-15 14:59:02 +00:00

llama_eagle.py

[Mypy] Fix mypy typing for L models (#54177)

2026-09-04 07:50:43 -04:00

llama.py

[Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514)

2026-09-05 11:45:34 +08:00

llava_next_video.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

llava_next.py

[MM] Further cleanup _apply_hf_processor_main (#53610)

2026-09-16 10:21:00 -07:00

llava_onevision2.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

llava_onevision.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

llava.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

longcat_flash_mtp.py

[Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep (#60080)

2026-10-06 08:47:02 +00:00

longcat_flash_ngram.py

[CI] Bump Transformers version to 5.18.0 (#59621)

2026-10-04 00:42:35 +00:00

longcat_flash.py

[Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep (#60080)

2026-10-06 08:47:02 +00:00

mamba2.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

mamba.py

[Mypy] Fix typing for M models (#54262)

2026-09-01 11:26:00 -04:00

midashenglm.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

mimo_audio.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

mimo_mtp.py

[XPU][Graph] Replace boolean-mask assignment with torch.where (#60529)

2026-10-08 17:13:48 +08:00

mimo_v2_mtp.py

[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797)

2026-10-14 10:17:58 +00:00

mimo_v2_omni.py

[Bugfix][MiMo] Declare embedding_fields so an EPD pair can serve images (#58938)

2026-09-29 23:33:01 +08:00

mimo_v2.py

[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797)

2026-10-14 10:17:58 +00:00

mimo.py

[Core][MRV2] Support eagle3 spec decode with pipeline parallel (#50514)

2026-09-05 11:45:34 +08:00

minicpm3.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

minicpm_eagle.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

minicpm.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

minicpmo.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

minicpmv4_6.py

[Model] Support MiniCPM-V 4.7 (#58674)

2026-10-11 10:21:10 +00:00

minicpmv4_7.py

[Model] Support MiniCPM-V 4.7 (#58674)

2026-10-11 10:21:10 +00:00

minicpmv.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

minimax_m2.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

mistral3.py

[Model] Support precomputed image embeddings for LightOnOCR (#61585)

2026-10-14 10:54:09 +00:00

mistral_eagle.py

[Mypy] Fix typing for M models (#54262)

2026-09-01 11:26:00 -04:00

mistral_large_3_eagle.py

[Bugfix][Spec Decode] Fix multimodal embedding merge in Llama 4 and Mistral Large 3 EAGLE drafters (#57633)

2026-10-09 01:44:50 +00:00

mistral_large_3.py

[MISC] Cleanup deprecated parameters (#53559)

2026-08-24 10:19:04 -07:00

mistral.py

[Mypy] Fix typing for M models (#54262)

2026-09-01 11:26:00 -04:00

mixtral.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

mllama4.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

modernbert.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

module_mapping.py

[Mypy] Fix typing for M models (#54262)

2026-09-01 11:26:00 -04:00

molmo2.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

molmo.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

moondream3.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

moonvit.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

moss_audio.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

moss_transcribe_diarize.py

[MM] Move get_dummy_processor_inputs into MM processor (#57967)

2026-09-21 09:39:20 -07:00

muse_glimmer.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

nano_nemotron_vl.py

[Bugfix] Load and apply Nemotron Super final vision LayerNorm (#61401)

2026-10-13 12:42:29 +00:00

nemotron_h_mtp.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

nemotron_h.py

[Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604)

2026-10-13 09:06:33 -07:00

nemotron_nas.py

Fix weight tying (#51665)

2026-08-20 10:31:55 +00:00

nemotron_parse.py

[Perf][Nemotron-Parse] Do image rescale/normalize on the device (#60500)

2026-10-14 18:33:22 +00:00

nemotron_vl.py

[MM] Enable device normalization for Llama Nemotron VL Embed/Rerank (#57928)

2026-09-29 08:33:52 +00:00

nemotron.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

nvlm_d.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

olmo_hybrid.py

[Mypy] Fix mypy typing for N/O models (#54142)

2026-09-18 12:50:01 +01:00

olmoe.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

openai_privacy_filter.py

[Model] Add support for openai/privacy-filter (#41026)

2026-06-28 16:18:22 -07:00

opencua.py

[Mypy] Fix mypy typing for N/O models (#54142)

2026-09-18 12:50:01 +01:00

openpangu_mtp.py

[Mypy] Fix mypy typing for N/O models (#54142)

2026-09-18 12:50:01 +01:00

openpangu_vl.py

[Bugfix] Fix the two multimodal root tests that fail on main (OpenPangu-VL embed merge, MiMo sink test fixture) (#58900)

2026-09-28 04:52:51 +00:00

openpangu.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

openvla.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

opt.py

[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172)

2026-09-30 12:19:33 +00:00

orion.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

ovis2_5.py

[BUGFIX] fix ovis2_5 multimodal tokens (#52623)

2026-09-25 07:18:54 +08:00

ovis.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00

paddleocr_vl.py

[Perf][Model] Batch PaddleOCR-VL vision encoder and projector (#61458)

2026-10-14 19:59:44 +08:00

paligemma.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

parakeet.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

param2moe.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

phi3.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

phi3v.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

phi4mm_audio.py

[Refactor] Remove dead code multiple places (#60120)

2026-10-10 15:41:10 +00:00

phi4mm_utils.py

[Security] Do not retain Phi-4 MM audio attention matrices (#60614)

2026-10-09 22:23:35 +08:00

phi4mm.py

[Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239)

2026-09-27 21:19:04 -04:00

phi4siglip.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

phimoe.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

pixtral.py

[Fusion] Support 3D activation quantization fusion and enable it for Pixtral (#59612)

2026-10-09 15:12:03 +08:00

plamo3.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

qianfan_ocr.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen2_5_omni_thinker.py

[Bugfix][Model][Rust Frontend] Qwen3_ASR: Handle ragged Qwen audio batches from Rust frontend (#60979)

2026-10-13 10:49:39 +00:00

qwen2_5_vl.py

[Multimodal] Avoid extra d2d for encoder cudagraph with fused input norm (#56711)

2026-09-28 21:04:01 +08:00

qwen2_audio.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen2_moe.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen2_rm.py

Remove unnecessary load_weights methods (#44589)

2026-06-29 01:52:23 -07:00

qwen2_vl.py

[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution (#56372)

2026-09-29 03:58:39 +00:00

qwen2.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen3_5_mtp.py

[Qwen3.5] Project the refusal direction out of the residual stream at runtime

2026-10-15 21:40:16 +07:00

qwen3_5.py

[Bugfix] Load checkpoint KV-cache scales in the Qwen3.5 family

2026-10-18 17:18:47 +00:00

qwen3_asr_forced_aligner.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen3_asr_realtime.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen3_asr.py

[Bugfix][Model][Rust Frontend] Qwen3_ASR: Handle ragged Qwen audio batches from Rust frontend (#60979)

2026-10-13 10:49:39 +00:00

qwen3_dflash2.py

[SpecDecode] Add LiLiCorr drafter (#57934)

2026-09-25 10:08:28 -04:00

qwen3_dflash.py

[Bugfix][RL] Load attention sinks through weight_loader for layerwise reload (#57797)

2026-10-14 10:17:58 +00:00

qwen3_dspark.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen3_eagle3.py

[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)

2026-09-26 14:31:51 +00:00

qwen3_moe.py

[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 (#59853)

2026-10-10 06:51:52 +00:00

qwen3_next_mtp.py

fix: keep the refusal-direction projection dynamo-safe

2026-10-15 22:56:06 +07:00

qwen3_next.py

[Qwen3.5] Project the refusal direction out of the residual stream at runtime

2026-10-15 21:40:16 +07:00

qwen3_omni_moe_thinker.py

[Model] Support Cloudflare/clef-omni on /v1/systemone (#61064)

2026-10-12 21:27:34 +08:00

qwen3_vl_moe.py

[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 (#59853)

2026-10-10 06:51:52 +00:00

qwen3_vl.py

[Offloader] Stream every multimodal tower parameter to host RAM

2026-10-16 16:07:53 +07:00

qwen3.py

[Model] Drop the 4x M-RoPE cache headroom for Qwen3-VL, Qwen3.5 and Qwen3.8 (#59853)

2026-10-10 06:51:52 +00:00

radio.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

registry.py

[Core][Feat] Pluggable KVCacheConfigBuilder for platform/model-specific KV cache planning (#53558)

2026-10-16 23:16:47 +00:00

rnj1.py

Fix weight tying (#51665)

2026-08-20 10:31:55 +00:00

roberta.py

[Model] Make the BERT/RoBERTa embedding class a class attribute (#59348)

2026-09-30 08:57:21 +00:00

rvl.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

sarvam.py

[Model][Spec Decode] Enable EAGLE3/DSpark pipeline parallelism for Sarvam MLA (#55902)

2026-10-02 04:28:19 +00:00

siglip2navit.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

siglip.py

[XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172)

2026-09-30 12:19:33 +00:00

skyworkr1v.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

smolvlm.py

…

solar.py

[Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550)

2026-09-24 14:14:10 +00:00

stablelm.py

[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)

2026-09-16 10:57:24 -07:00

step1.py

Remove unnecessary load_weights methods (#44589)

2026-06-29 01:52:23 -07:00

step3_text.py

[MoE] Use GateLinear for all MoE models (#58234)

2026-09-23 05:40:13 +00:00

step3_vl.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

step3p5_mtp.py

[Bugfix][Spec Decode] Per-module LM heads for multi-layer MTP on Model Runner V2 (#58921)

2026-10-01 01:09:14 -07:00

step3p5.py

[Refactor] Remove dead code multiple places (#60120)

2026-10-10 15:41:10 +00:00

step3p7.py

[Mypy] Fix typing for R/S models (#54157)

2026-09-10 09:43:08 -04:00

step_vl.py

Add pydocstyle to the ruff rules (#52136)

2026-09-16 16:44:29 +00:00

telechat2.py

[Mypy Fix] Mypy fix for "vllm/model_executor/models/[tT]" (#53466)

2026-08-26 14:12:11 +00:00

teleflm.py

[Refactor] Remove dead code multiple places (#58002)

2026-09-22 16:36:03 -04:00

terratorch.py

[Multimodal] Type dummy options per modality (#57576)

2026-09-18 17:29:11 +01:00

ultravox.py

[Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239)

2026-09-27 21:19:04 -04:00

unlimited_ocr.py

[Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239)

2026-09-27 21:19:04 -04:00

utils.py

[Bugfix][weight loader] Stop warning when a parameter only widens the weight (#60020)

2026-10-12 14:33:31 -04:00

vision.py

[Mypy] Fix mypy typing for Voxtral and vision models (#58251)

2026-09-28 15:52:50 -04:00

voxtral_realtime.py

[Bugfix][Model] Voxtral realtime: fix boot OOM and engine crash at max_model_len (#45022)

2026-10-09 17:40:29 +08:00

voxtral.py

[Mypy] Fix mypy typing for Voxtral and vision models (#58251)

2026-09-28 15:52:50 -04:00

voyage.py

[Model][LoRA] Enable LoRA support for VoyageQwen3BidirectionalEmbedModel (#57708)

2026-09-20 09:50:00 +00:00

whisper_causal.py

[Mypy] Fix mypy typing for Whisper models (#58254)

2026-09-29 17:10:28 +01:00

whisper_utils.py

…

whisper.py

[CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kernel (#58268)

2026-09-30 13:52:53 +08:00

zamba2.py

[Core][Mamba] Remove leftovers of mamba_cache_mode="all" (#60043)

2026-10-13 00:58:59 -07:00