Eugene KhvedchenyaandRoger Wang 392edee34a EVS Support (Video tokens pruning) (#22980) ...

Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

2025-09-26 11:54:54 +08:00

..

__init__.py

[feat]: Create interface for model-specific M-RoPE (#24194)

2025-09-18 19:18:56 +00:00

adapters.py

Enable conversion of multimodal models to pooling tasks (#24451)

2025-09-12 03:30:41 +00:00

aimv2.py

[Misc] Use config definitions from Transformers library (#21913)

2025-08-08 23:10:51 -07:00

apertus.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

arcee.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

arctic.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

aria.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

aya_vision.py

[Misc] Improve type annotations for jsontree (#25577)

2025-09-24 22:49:58 +08:00

baichuan.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

bailing_moe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

bamba.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

bert_with_rope.py

[Kernel] Delegate construction of FusedMoEQuantConfig to FusedMoEMethodBase subclasses (#22537)

2025-09-17 17:43:31 -06:00

bert.py

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872)

2025-09-18 23:22:01 +08:00

blip2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

blip.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

bloom.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

chameleon.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

chatglm.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

clip.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

cohere2_vision.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

commandr.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

config.py

[V0 Deprecation] Remove max_seq_len_to_capture (#25543)

2025-09-24 01:51:39 -07:00

dbrx.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

deepseek_eagle.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

deepseek_mtp.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

deepseek_v2.py

[Kernel] Support DCP for Triton backend (#25132)

2025-09-24 18:09:34 -07:00

deepseek_vl2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

deepseek.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

dots1.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

dots_ocr.py

[Model] Improve DotsOCRForCausalLM (#25466)

2025-09-25 07:58:08 +08:00

ernie45_moe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

ernie45_vl_moe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

ernie45_vl.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

ernie45.py

[Ernie 4.5] Name Change for Base 0.3B Model (#21735)

2025-07-28 12:22:32 +00:00

ernie_mtp.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

exaone4.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

exaone.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

fairseq2_llama.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

falcon_h1.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

falcon.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

fuyu.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gemma2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gemma3_mm.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gemma3.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gemma3n_mm.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gemma3n.py

[KV sharing] Re-land Gemma3n model changes from #22628 (#24357)

2025-09-23 19:25:34 -07:00

gemma.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

glm4_1v.py

[Misc] Move DP for ViT code inside model executor dir (#25459)

2025-09-23 09:20:52 +00:00

glm4_moe_mtp.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

glm4_moe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

glm4.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

glm4v.py

[Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053)

2025-08-18 09:52:00 +00:00

glm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

gpt2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gpt_bigcode.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gpt_j.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gpt_neox.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

gpt_oss.py

Enable Eagle3 speculative decoding for GPT-OSS model (#25246)

2025-09-22 08:50:39 +00:00

granite_speech.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

granite.py

[CI/Build] Fix and re-enable v1 PP test on CI (#25496)

2025-09-23 21:58:25 +00:00

granitemoe.py

[CI/Build] Fix and re-enable v1 PP test on CI (#25496)

2025-09-23 21:58:25 +00:00

granitemoehybrid.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

granitemoeshared.py

[CI/Build] Fix and re-enable v1 PP test on CI (#25496)

2025-09-23 21:58:25 +00:00

gritlm.py

[Misc] Simplify PoolerOutput and move to v1/outputs (#25629)

2025-09-25 06:47:03 +00:00

grok1.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

h2ovl.py

[Core] Simplify and unify mm uuid handling & auto-generated mm hash overrides processing. (#24271)

2025-09-09 21:36:09 -07:00

hunyuan_v1.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

hyperclovax_vision.py

[Misc]] Move processing context to multimodal directory (#25548)

2025-09-24 08:15:00 +00:00

idefics2_vision_model.py

[Misc] Move DP for ViT code inside model executor dir (#25459)

2025-09-23 09:20:52 +00:00

idefics3.py

[Bugfix] Fix idefics3 tie_word_embeddings (#25454)

2025-09-23 10:06:48 +00:00

interfaces_base.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

interfaces.py

EVS Support (Video tokens pruning) (#22980)

2025-09-26 11:54:54 +08:00

intern_vit.py

[Misc] Move DP for ViT code inside model executor dir (#25459)

2025-09-23 09:20:52 +00:00

internlm2_ve.py

[Models] Improve iteration over layers (#19497)

2025-08-29 09:26:34 +08:00

internlm2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

interns1_vit.py

Feature/vit attention unification# 23880 (#23978)

2025-09-10 06:10:14 -07:00

interns1.py

[Bugfix] Fix InternS1 video processing after Transformers v4.56 (#25644)

2025-09-25 14:46:04 +00:00

internvl.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

jais.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

jamba.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

jina_vl.py

[Model] Systematic support for fp32 head, pooling models part (#23810)

2025-09-09 07:29:50 -07:00

keye_vl1_5.py

[Docs] Fix warnings in mkdocs build (continued) (#24791)

2025-09-13 00:13:44 -07:00

keye.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

kimi_vl.py

[Misc] Move DP for ViT code inside model executor dir (#25459)

2025-09-23 09:20:52 +00:00

lfm2.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

llama4_eagle.py

[Perf] Optimize memory peak during EAGLE model loading. (#24585)

2025-09-19 03:31:16 +00:00

llama4.py

[Kernels] Overlap shared experts with send/recv (#23273)

2025-09-03 12:35:18 -04:00

llama_eagle3.py

[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406)

2025-09-23 15:44:35 -04:00

llama_eagle.py

[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406)

2025-09-23 15:44:35 -04:00

llama.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

llava_next_video.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

llava_next.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

llava_onevision.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

llava.py

[Misc] Improve type annotations for jsontree (#25577)

2025-09-24 22:49:58 +08:00

longcat_flash_mtp.py

[Model] Add LongCat-Flash (#23991)

2025-09-24 21:53:40 -07:00

longcat_flash.py

[Model] Add LongCat-Flash (#23991)

2025-09-24 21:53:40 -07:00

mamba2.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

mamba.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

medusa.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

midashenglm.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

mimo_mtp.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

mimo.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

minicpm3.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

minicpm_eagle.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

minicpm.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

minicpmo.py

Migrate MiniCPMOAudioInputs to TensorSchema (#21847)

2025-08-22 16:43:29 +08:00

minicpmv.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

minimax_text_01.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

minimax_vl_01.py

[Misc] Improve type annotations for jsontree (#25577)

2025-09-24 22:49:58 +08:00

mistral3.py

[Misc]] Move processing context to multimodal directory (#25548)

2025-09-24 08:15:00 +00:00

mixtral.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

mllama4.py

[Misc]] Move processing context to multimodal directory (#25548)

2025-09-24 08:15:00 +00:00

mlp_speculator.py

[V0 Deprecation] Remove V0 Sequence class & Sampler (#25332)

2025-09-21 08:52:15 -07:00

modernbert.py

[Model] Systematic support for fp32 head, pooling models part (#23810)

2025-09-09 07:29:50 -07:00

module_mapping.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

molmo.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

moonvit.py

[Docs] Fix warnings in mkdocs build (continued) (#24791)

2025-09-13 00:13:44 -07:00

motif.py

[Model] New model support for Motif-1-Tiny (#23414)

2025-09-10 23:29:40 -07:00

mpt.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

nano_nemotron_vl.py

[Model] rename NemotronH_Nano_VL -> NemotronH_Nano_VL_V2 (#25708)

2025-09-25 16:10:29 -07:00

nemotron_h.py

[V1] Remove V0 code paths for Hybrid models (#25400)

2025-09-23 08:26:13 -07:00

nemotron_nas.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

nemotron_vl.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

nemotron.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

nvlm_d.py

[Refactor] Define MultiModalKwargsItems separate from MultiModalKwargs (#23053)

2025-08-18 09:52:00 +00:00

olmo2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

olmo.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

olmoe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

opt.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

orion.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

ovis2_5.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

ovis.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

paligemma.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

persimmon.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

phi3.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

phi3v.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

phi4_multimodal.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

phi4mm_audio.py

[Doc]: fixing doc typos (#24635)

2025-09-10 23:19:28 -07:00

phi4mm_utils.py

[Docs] Fix warnings in mkdocs build (continued) (#24092)

2025-09-10 06:23:28 -07:00

phi4mm.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

phi.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

phimoe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

pixtral.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

plamo2.py

Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512)

2025-09-23 22:48:40 +00:00

qwen2_5_omni_thinker.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

qwen2_5_vl.py

EVS Support (Video tokens pruning) (#22980)

2025-09-26 11:54:54 +08:00

qwen2_audio.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

qwen2_moe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

qwen2_rm.py

[Model] Systematic support for fp32 head, pooling models part (#23810)

2025-09-09 07:29:50 -07:00

qwen2_vl.py

[Bugfix] Fix Qwen3-VL max_num_video_tokens calculation for video profiling (#25648)

2025-09-25 18:36:01 +08:00

qwen2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

qwen3_moe.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

qwen3_next_mtp.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

qwen3_next.py

Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512)

2025-09-23 22:48:40 +00:00

qwen3_vl_moe.py

[Bugfix] Fix Qwen3-VL-MoE weight loading for EP (#25300)

2025-09-20 00:04:05 -07:00

qwen3_vl.py

[Bugfix] Fix Qwen3-VL max_num_video_tokens calculation for video profiling (#25648)

2025-09-25 18:36:01 +08:00

qwen3.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

qwen_vl.py

Migrate Qwen inputs to TensorSchema (#23473)

2025-08-28 10:43:26 +08:00

qwen.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

radio.py

Add RADIO Vision Encoder Support to vLLM (#24595)

2025-09-17 05:53:30 -07:00

registry.py

[Model] rename NemotronH_Nano_VL -> NemotronH_Nano_VL_V2 (#25708)

2025-09-25 16:10:29 -07:00

roberta.py

[Model] Systematic support for fp32 head, pooling models part (#23810)

2025-09-09 07:29:50 -07:00

rvl.py

[Model][VLM] Support R-4B Model (#23246)

2025-08-21 04:08:52 +00:00

seed_oss.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

siglip2navit.py

[Docs] Fix warnings in mkdocs build (continued) (#24791)

2025-09-13 00:13:44 -07:00

siglip.py

Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771)

2025-09-19 22:40:33 +00:00

skyworkr1v.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

smolvlm.py

[Misc] Automatically resolve HF processor init kwargs (#22005)

2025-07-31 22:44:10 -07:00

solar.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

stablelm.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

starcoder2.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

step3_text.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

step3_vl.py

[Misc] Move DP for ViT code inside model executor dir (#25459)

2025-09-23 09:20:52 +00:00

swin.py

[New Model]Donut model (#23229)

2025-08-24 12:52:24 +00:00

tarsier.py

[Misc] Improve type annotations for jsontree (#25577)

2025-09-24 22:49:58 +08:00

telechat2.py

[CI/Build] Fix registry tests (#21934)

2025-07-30 09:10:41 -07:00

teleflm.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

terratorch.py

[Core] Simplify and unify mm uuid handling & auto-generated mm hash overrides processing. (#24271)

2025-09-09 21:36:09 -07:00

transformers.py

[mypy] Further improve MM type annotations (#25654)

2025-09-25 10:57:36 +00:00

ultravox.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

utils.py

[Model] Add LongCat-Flash (#23991)

2025-09-24 21:53:40 -07:00

vision.py

[Misc] Move DP for ViT code inside model executor dir (#25459)

2025-09-23 09:20:52 +00:00

voxtral.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

whisper.py

[V0 Deprecation] Remove V0 sampling metadata (#25345)

2025-09-21 10:37:11 -07:00

zamba2.py

[Docs] Enable fail_on_warning for the docs build in CI (#25580)

2025-09-24 19:30:33 +00:00