3 people 706ff13224 [Model] Adds support for SlimMoE models Phi-tiny-MoE-instruct (#20286) ...

Signed-off-by: Zichong Li <t-lizichong@microsoft.com@Reasoning-H100-VM3.drbuo4tcjzruhloch3eo0b25ef.cx.internal.cloudapp.net>
Co-authored-by: Zichong Li <t-lizichong@microsoft.com@Reasoning-H100-VM3.drbuo4tcjzruhloch3eo0b25ef.cx.internal.cloudapp.net>
Co-authored-by: Isotr0py <2037008807@qq.com>

2025-07-02 12:54:12 +00:00

..

__init__.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

adapters.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

aimv2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

arctic.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

aria.py

[Core] More fixes to MultiModalEmbeddings type handling (#19715)

2025-06-18 22:48:29 +00:00

aya_vision.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

baichuan.py

[CI] change spell checker from codespell to typos (#18711)

2025-06-11 19:57:10 -07:00

bamba.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

bart.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

bert_with_rope.py

[Model][1/N] Automatic conversion of CrossEncoding model (#20012)

2025-06-26 21:10:04 -07:00

bert.py

Support embedding models in V1 (#16188)

2025-06-18 21:36:33 -07:00

blip2.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

blip.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

bloom.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

chameleon.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

chatglm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

clip.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

commandr.py

[Model] use AutoWeightsLoader for commandr (#19399)

2025-06-10 22:42:21 +00:00

config.py

[Model][1/N] Automatic conversion of CrossEncoding model (#20012)

2025-06-26 21:10:04 -07:00

constant_size_cache.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

dbrx.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

deepseek_mtp.py

[V1][Speculative Decoding] Fix DeepSeek MTP (#20022)

2025-06-25 08:41:02 -07:00

deepseek_v2.py

[BUGFIX][DEEPSEEK][MODEL_LOAD] fix w13, w2 weight not initialized assert (#20202)

2025-06-29 19:46:19 -07:00

deepseek_vl2.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

deepseek.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

dots1.py

[Model] support dots1 (#18254)

2025-06-29 19:34:36 -07:00

eagle.py

[Bugfix][2/n] Fix speculative decoding CI - Fix test_ngram_e2e_greedy_correctness (#19644)

2025-06-14 21:15:41 -07:00

ernie45_moe.py

[Model] Add Ernie4.5 and Ernie4.5MoE Model Support (#20220)

2025-07-02 03:37:01 -07:00

ernie45.py

[Model] Add Ernie4.5 and Ernie4.5MoE Model Support (#20220)

2025-07-02 03:37:01 -07:00

exaone.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

fairseq2_llama.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

falcon_h1.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

falcon.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

florence2.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

fuyu.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

gemma2.py

Mark invariant normalizer in Gemma as non-persistent (#19788)

2025-06-18 22:56:03 -07:00

gemma3_mm.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

gemma3.py

Mark invariant normalizer in Gemma as non-persistent (#19788)

2025-06-18 22:56:03 -07:00

gemma3n.py

Gemma3n (Text-only) (#20134)

2025-06-27 07:16:26 +00:00

gemma.py

Mark invariant normalizer in Gemma as non-persistent (#19788)

2025-06-18 22:56:03 -07:00

glm4_1v.py

Add GLM4.1V model (Draft) (#19331)

2025-07-01 12:48:26 +00:00

glm4.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

glm4v.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

glm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

gpt2.py

[Model] GPT2ForSequenceClassification model (#19663)

2025-06-20 12:07:41 +00:00

gpt_bigcode.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

gpt_j.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

gpt_neox.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

granite_speech.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

granite.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

granitemoe.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

granitemoehybrid.py

[Bugfix] Correct behavior of GraniteMoeHybrid for TensorParallel execution (#20137)

2025-06-28 08:16:41 -07:00

granitemoeshared.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

gritlm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

grok1.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

h2ovl.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

hunyuan_v1_moe.py

[Model]Add Tencent HunYuanMoEV1 Model Support (#20114)

2025-07-01 07:28:13 -07:00

idefics2_vision_model.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

idefics3.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

interfaces_base.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

interfaces.py

[Quant] [Bugfix] Fix quantization config matching with hf_to_vllm_mapper (#20046)

2025-07-01 19:20:34 +09:00

intern_vit.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

internlm2_ve.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

internlm2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

internvl.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

jais.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

jamba.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

keye.py

[Bugfix] Keye-VL compatibility with tok_kwargs (#20058) (#20353)

2025-07-01 23:46:59 -07:00

kimi_vl.py

[Core] More fixes to MultiModalEmbeddings type handling (#19715)

2025-06-18 22:48:29 +00:00

llama4.py

[Llama4] Update attn_temperature_tuning (#19997)

2025-06-24 22:42:53 -04:00

llama_eagle3.py

[Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033)

2025-06-05 19:10:08 -07:00

llama_eagle.py

[Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033)

2025-06-05 19:10:08 -07:00

llama.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

llava_next_video.py

[Core] More fixes to MultiModalEmbeddings type handling (#19715)

2025-06-18 22:48:29 +00:00

llava_next.py

[Core] More fixes to MultiModalEmbeddings type handling (#19715)

2025-06-18 22:48:29 +00:00

llava_onevision.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

llava.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

mamba2.py

[v1] Support mamba2 (#19327)

2025-06-18 20:34:15 +00:00

mamba_cache.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

mamba.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

medusa.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

mimo_mtp.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

mimo.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

minicpm3.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

minicpm_eagle.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

minicpm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

minicpmo.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

minicpmv.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

minimax_cache.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

minimax_text_01.py

[Model] Fix minimax model cache & lm_head precision (#19592)

2025-06-13 12:08:20 +00:00

minimax_vl_01.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

mistral3.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

mixtral_quant.py

[CI] change spell checker from codespell to typos (#18711)

2025-06-11 19:57:10 -07:00

mixtral.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

mllama4.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

mllama.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

mlp_speculator.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

modernbert.py

[Fix] Support cls pooling in ModernBertPooler (#20067)

2025-06-25 15:07:45 -04:00

module_mapping.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

molmo.py

[Core] More fixes to MultiModalEmbeddings type handling (#19715)

2025-06-18 22:48:29 +00:00

moonvit.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

mpt.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

nemotron_h.py

[Model] Optimize nemotron_h implementation (#19249)

2025-06-06 10:05:14 +00:00

nemotron_nas.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

nemotron.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

nvlm_d.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

olmo2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

olmo.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

olmoe.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

opt.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

orion.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

ovis.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

paligemma.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

persimmon.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

phi3_small.py

[CI] change spell checker from codespell to typos (#18711)

2025-06-11 19:57:10 -07:00

phi3.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

phi3v.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

phi4mm_audio.py

[CI] change spell checker from codespell to typos (#18711)

2025-06-11 19:57:10 -07:00

phi4mm_utils.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

phi4mm.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

phi.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

phimoe.py

[Model] Adds support for SlimMoE models Phi-tiny-MoE-instruct (#20286)

2025-07-02 12:54:12 +00:00

pixtral.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

plamo2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

prithvi_geospatial_mae.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

qwen2_5_omni_thinker.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

qwen2_5_vl.py

[Quant] [Bugfix] Fix quantization config matching with hf_to_vllm_mapper (#20046)

2025-07-01 19:20:34 +09:00

qwen2_audio.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

qwen2_moe.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

qwen2_rm.py

[Doc] Update V1 status for decoder-only embedding models (#19952)

2025-06-23 09:31:06 +00:00

qwen2_vl.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

qwen2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

qwen3_moe.py

[Bugfix] Skip loading extra parameters for modelopt Qwen3 MoE model (#19598)

2025-06-30 18:21:56 +09:00

qwen3.py

[Model][1/N] Automatic conversion of CrossEncoding model (#20012)

2025-06-26 21:10:04 -07:00

qwen_vl.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

qwen.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

registry.py

[Model] Add Ernie4.5 and Ernie4.5MoE Model Support (#20220)

2025-07-02 03:37:01 -07:00

roberta.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

siglip.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

skyworkr1v.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

smolvlm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

solar.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

stablelm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

starcoder2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

tarsier.py

[Core] More fixes to MultiModalEmbeddings type handling (#19715)

2025-06-18 22:48:29 +00:00

telechat2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

teleflm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

transformers.py

[Quant] [Bugfix] Fix quantization config matching with hf_to_vllm_mapper (#20046)

2025-07-01 19:20:34 +09:00

ultravox.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

utils.py

[Quant] [Bugfix] Fix quantization config matching with hf_to_vllm_mapper (#20046)

2025-07-01 19:20:34 +09:00

vision.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

whisper.py

[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058)

2025-06-30 17:26:49 +00:00

zamba2.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00