Jee Jee LiandIsotr0py a655eb3025 [Misc]Add BNB quantization for Qwen2VL (#11719) ...

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

2025-01-03 15:19:02 -07:00

..

__init__.py

[Misc] Rename embedding classes to pooling (#10801)

2024-12-01 14:36:51 +08:00

adapters.py

[Model] Automatic conversion of classification and reward models (#11469)

2024-12-24 18:22:22 +00:00

arctic.py

[model][utils] add extract_layer_index utility function (#10599)

2024-11-23 22:22:54 -08:00

aria.py

[VLM] Move supported limits and max tokens to merged multi-modal processor (#11669)

2025-01-01 15:44:42 +00:00

baichuan.py

[Misc]Further reduce BNB static variable (#10597)

2024-11-26 22:54:12 -08:00

bart.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

bert.py

[Misc] Move weights mapper (#11443)

2024-12-24 13:05:25 +00:00

blip2.py

[VLM] Move supported limits and max tokens to merged multi-modal processor (#11669)

2025-01-01 15:44:42 +00:00

blip.py

[V1][VLM] V1 support for selected single-image models. (#11632)

2024-12-31 21:17:22 +00:00

bloom.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

chameleon.py

[VLM] Move supported limits and max tokens to merged multi-modal processor (#11669)

2025-01-01 15:44:42 +00:00

chatglm.py

[VLM] Abstract out multi-modal data parsing in merged processor (#11620)

2024-12-30 15:01:35 +00:00

clip.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

commandr.py

[bugfix] interleaving sliding window for cohere2 model (#11583)

2024-12-28 16:55:42 +00:00

dbrx.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

decilm.py

[Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327)

2024-11-18 09:07:46 +08:00

deepseek_v2.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

deepseek_v3.py

Deepseek v3 (#11502)

2024-12-26 16:09:44 -08:00

deepseek.py

[model][utils] add extract_layer_index utility function (#10599)

2024-11-23 22:22:54 -08:00

eagle.py

[V1] Refactor model executable interface for all text-only language models (#10374)

2024-11-17 05:18:46 +00:00

exaone.py

[Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688)

2025-01-02 21:11:39 +00:00

falcon.py

[Misc]Further reduce BNB static variable (#10597)

2024-11-26 22:54:12 -08:00

florence2.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

fuyu.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

gemma2.py

[Bugfix] Add kv cache scales to gemma2.py (#11269)

2024-12-23 19:30:45 +00:00

gemma.py

[Misc]Further reduce BNB static variable (#10597)

2024-11-26 22:54:12 -08:00

glm4_vision_encoder.py

[Model] Consolidate ViTs attention implementation without mask (#10893)

2024-12-04 18:11:08 +00:00

glm.py

[Model] Added GLM-4 series hf format model support vllm==0.6.4 (#10561)

2024-11-28 14:53:31 +00:00

gpt2.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

gpt_bigcode.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

gpt_j.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

gpt_neox.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

granite.py

[Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688)

2025-01-02 21:11:39 +00:00

granitemoe.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

gritlm.py

[Frontend] Separate pooling APIs in offline inference (#11129)

2024-12-13 10:40:07 +00:00

h2ovl.py

[1/N] Initial prototype for multi-modal processor (#10044)

2024-11-13 12:39:03 +00:00

idefics2_vision_model.py

[V1][VLM] V1 support for selected single-image models. (#11632)

2024-12-31 21:17:22 +00:00

idefics3.py

[Bugfix] Fix Idefics3 fails during multi-image inference (#11080)

2024-12-11 01:27:07 -08:00

interfaces_base.py

[Misc] Rename embedding classes to pooling (#10801)

2024-12-01 14:36:51 +08:00

interfaces.py

[Model] PP support for Mamba-like models (#10992)

2024-12-10 21:53:37 -05:00

intern_vit.py

[Model] Consolidate ViTs attention implementation without mask (#10893)

2024-12-04 18:11:08 +00:00

internlm2_ve.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

internlm2.py

[Model] Support InternLM2 Reward models (#11571)

2024-12-28 06:14:10 +00:00

internvl.py

[V1][VLM] Fix edge case bug for InternVL2 (#11165)

2024-12-13 11:09:30 +00:00

jais.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

jamba.py

[MODEL] LoRA support for Jamba model (#11209)

2024-12-27 17:58:21 +00:00

llama.py

[Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688)

2025-01-02 21:11:39 +00:00

llava_next_video.py

[Model] Replace embedding models with pooling adapter (#10769)

2024-12-01 08:02:54 +08:00

llava_next.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

llava_onevision.py

[Model] Replace embedding models with pooling adapter (#10769)

2024-12-01 08:02:54 +08:00

llava.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

mamba_cache.py

[Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189)

2024-10-16 12:12:43 -04:00

mamba.py

[MODEL] LoRA support for Jamba model (#11209)

2024-12-27 17:58:21 +00:00

medusa.py

[model] Reduce medusa weight (#10454)

2024-11-20 06:05:55 +00:00

minicpm3.py

[Model] Clean up MiniCPMV (#10751)

2024-11-29 04:47:06 +00:00

minicpm.py

[Model] support bitsandbytes quantization with minicpm model (#10842)

2024-12-03 03:06:41 +00:00

minicpmv.py

[Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel (#11631)

2024-12-30 18:51:04 +00:00

mixtral_quant.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

mixtral.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

mllama.py

[Misc]Further reduce BNB static variable (#10597)

2024-11-26 22:54:12 -08:00

mlp_speculator.py

[Bugfix] Fix Mamba model initialization and MLP Speculator weights loading (#10456)

2024-11-20 05:04:05 +00:00

module_mapping.py

[Model][LoRA]LoRA support added for MiniCPMV2.5 (#7199)

2024-09-29 06:59:45 +00:00

molmo.py

[Model][LoRA]LoRA support added for MolmoForCausalLM (#11439)

2024-12-30 17:33:06 -08:00

mpt.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

nemotron.py

[Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975)

2024-12-07 17:20:49 +00:00

nvlm_d.py

[Bugfix] Fix _init_vision_model in NVLM_D model (#9611)

2024-10-23 14:09:04 +00:00

olmo2.py

[Model] Add OLMo November 2024 model (#10503)

2024-11-25 17:26:40 -05:00

olmo.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

olmoe.py

[model][utils] add extract_layer_index utility function (#10599)

2024-11-23 22:22:54 -08:00

opt.py

[Misc]Further reduce BNB static variable (#10597)

2024-11-26 22:54:12 -08:00

orion.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

paligemma.py

PaliGemma 2 support (#11142)

2024-12-13 07:40:07 +00:00

persimmon.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

phi3_small.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

phi3.py

[Misc]Further reduce BNB static variable (#10597)

2024-11-26 22:54:12 -08:00

phi3v.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

phi.py

[Misc]Further reduce BNB static variable (#10597)

2024-11-26 22:54:12 -08:00

phimoe.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

pixtral.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

qwen2_audio.py

[VLM] Move supported limits and max tokens to merged multi-modal processor (#11669)

2025-01-01 15:44:42 +00:00

qwen2_moe.py

[model][utils] add extract_layer_index utility function (#10599)

2024-11-23 22:22:54 -08:00

qwen2_rm.py

[Misc] Add uninitialized params tracking for AutoWeightsLoader (#10327)

2024-11-18 09:07:46 +08:00

qwen2_vl.py

[Misc]Add BNB quantization for Qwen2VL (#11719)

2025-01-03 15:19:02 -07:00

qwen2.py

[Model] Automatic conversion of classification and reward models (#11469)

2024-12-24 18:22:22 +00:00

qwen.py

[VLM] Support caching in merged multi-modal processor (#11396)

2024-12-27 17:22:48 +00:00

registry.py

[Model] Whisper model implementation (#11280)

2025-01-03 16:39:19 +08:00

roberta.py

[Model] Enable optional prefix when loading embedding models (#10639)

2024-11-25 18:14:33 +00:00

siglip.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

solar.py

[Bugfix] Change kv scaling factor by param json on nvidia gpu (#11688)

2025-01-02 21:11:39 +00:00

stablelm.py

[torch.compile] support all attention backends (#10558)

2024-11-22 14:04:42 -08:00

starcoder2.py

[Bugfix] Update starcoder2 to remap k/v scale names for kv_cache quantization (#11148)

2024-12-13 05:07:20 +00:00

telechat2.py

[Bugfix] Fix TeleChat2ForCausalLM weights mapper (#11546)

2024-12-27 10:39:15 +00:00

ultravox.py

[VLM] Move supported limits and max tokens to merged multi-modal processor (#11669)

2025-01-01 15:44:42 +00:00

utils.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

vision.py

[VLM] Merged multi-modal processor for LLaVA-NeXT (#11682)

2025-01-02 16:39:27 +00:00

whisper.py

[Model] Whisper model implementation (#11280)

2025-01-03 16:39:19 +08:00