MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
Isotr0py
522101f6ed
[Multimodal] Avoid extra d2d for encoder cudagraph with fused input norm ( #56711 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-09-28 21:04:01 +08:00
Ashraf Bhuiyan
5113cf9d16
[Mypy] Fix mypy typing for Qwen and Qianfan models ( #58046 )
...
Signed-off-by: Ashraf Bhuiyan <mbhuiyan@redhat.com >
2026-09-26 14:31:51 +00:00
8a269919f4
[MM] Add Triton kernel for mm_input_normal. ( #56798 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-25 04:31:20 +00:00
Harry Mellor and Claude Opus 5.5
d881812500
[Chore] Use Transformers v5 names and drop redundant processor use_fast ( #58550 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 14:14:10 +00:00
Ting SUN and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a33b3bac5d
[Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE ( #47736 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-22 05:56:12 +00:00
cjackal
403f182c02
[MM] Keep raw pixels through dp-sharded ViT path ( #56872 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
2026-09-16 18:47:27 +00:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
linitra24
9d4d9aa5bc
[LoRA][Refactor] Unify multimodal LoRA token count hooks ( #55071 )
...
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io >
2026-09-14 13:19:15 +00:00
Yunzhu Lu and Isotr0py
442d36031c
[MM][CG] Enable encoder CUDA Graph for MiniCPM-V ( #42785 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 06:28:48 +00:00
22d95d1adc
[Bugfix] Make mm_device_do_normalize encoder-cudagraph safe ( #55370 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 01:43:22 +08:00
Nick Hill and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
a566ea7e8f
[Perf] Avoid more h2d copies from non-pinned tensors ( #54660 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-01 20:37:09 -07:00
b016ed8ea3
[Bugfix][Multimodal] Release Qwen2.5-VL and Qwen3-VL RoPE caches with the model ( #54346 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-30 11:10:29 +08:00
Nick Hill
6c18a54648
[Perf] Avoid h2d copies from non-pinned CPU tensors ( #54299 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-08-29 12:06:30 +08:00
Cyrus Leung
0b19ebcacd
[MM] Simplify _apply_hf_processor_main ( #53275 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-22 01:49:37 -07:00
linitra24
e85d1b69cf
[Bugfix][LoRA] Use an explicit capability flag for tower connector LoRA ( #53092 )
...
Signed-off-by: linitra24 <Joy25810@foxmail.com >
2026-08-21 09:45:54 +00:00
Cyrus Leung
b00f475f09
[MM] Remove text components from ProcessorInputs ( #53093 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-20 20:42:53 -07:00
Nick Hill and Claude Opus 4.8
615d4cfade
[Core] Check for GPU<->CPU syncs during CI ( #43107 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-14 17:42:05 -07:00
wang.yuqi and Cyrus Leung
2fa490470d
[Model] Fused mm preprocess normalisation on the Device ( #50411 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-06 03:11:14 -07:00
nvbfalk
60915c972c
[Feature] Add VidCom2 video token pruning ( #47750 )
...
Signed-off-by: Benedikt Falk <bfalk@nvidia.com >
2026-07-28 03:20:17 +00:00
Harry Mellor
1ab9522935
Remove more unnecessary load_weights methods ( #47058 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-30 15:22:16 +01:00
Nick Hill and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
84c62e1cbd
[Model Runner V2][MM] Support EVS ( #46535 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 10:18:56 -04:00
Nick Hill
7df3d7dada
[Core] Ensure memory is pinned prior to async h2d copy ( #45424 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-20 20:02:24 -07:00
ad32608e24
[MM][Perf][CG] Support dual-path ViT full CUDA graph for DeepSeek-OCR ( #43586 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-06-16 04:35:20 -07:00
Weida Hong
dfe8ba7c80
Adjust design around encoder_cudagraph_forward ( #42288 )
...
Signed-off-by: Weida Hong <wdhongtw@google.com >
2026-05-29 03:02:52 +00:00
Hua Huang
9006204e90
[MM][CG] Avoid over-padding Qwen2.5-VL encoder cudagraph window metadata ( #42796 )
...
Signed-off-by: Hua Huang <huah@nvidia.com >
2026-05-28 11:22:56 -07:00
Andreas Karatzas
5bdb181df5
[ROCm][CI] Fix ROCm multimodal Qwen2.5-VL activation compile and Phi4MM ragged image mask handling ( #43647 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-26 19:53:34 -07:00
Hua Huang and Isotr0py
a0be71ee47
[MM] Enable FlashInfer metadata support for Qwen2.5-VL vision attention ( #42787 )
...
Signed-off-by: Hua Huang <huah@nvidia.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-05-23 16:08:40 +00:00
Isotr0py
f0feb15e7f
[Multimodal] Simplify ViT CUDA graph interfaces ( #41234 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-05-22 22:31:00 +08:00
Nick Hill
b82e908b4c
[Perf][4/n] Eliminate various GPU<->CPU syncs ( #42347 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-19 10:35:54 -04:00
John Calderon
964a4bc2a5
[MM][CG] Support ViT CG for Qwen2.5-VL ( #40830 )
...
Signed-off-by: John Calderon <jcalderon@nvidia.com >
2026-05-02 11:10:14 +08:00
Cyrus Leung
502c41a8f6
[Model] Use helper function to run MM processors with token inputs (where applicable) ( #38018 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-26 16:44:04 +08:00
Lucas Kabela
7769b58307
[torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict ( #37345 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-03-19 17:26:12 +00:00
Lucas Kabela
714c6e0eab
[torch.compile][BE] Modify cudagraph callable to check for is_forward_context_set ( #36288 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-03-16 19:42:34 +00:00
Benjamin Chislett
8b346309a5
[Refactor] Consolidate SupportsEagle ( #36063 )
...
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com >
2026-03-13 23:22:40 +00:00
Lucas Kabela
3fd03f1ec2
[BE] Rename should_torch_compile_mm_vit to should_torch_compile_mm_encoder ( #36281 )
...
Signed-off-by: Lucas Kabela <lucaskabela@meta.com >
2026-03-09 18:22:05 +00:00
William Zhang
70c73df69e
[Bugfix] Fix EVS implementation for Qwen3 VL ( #33607 )
...
Signed-off-by: 2ez4bz <133824995+2ez4bz@users.noreply.github.com >
2026-03-04 02:18:11 +00:00
9c3fe9936b
Flashinfer cuDNN backend for Qwen3 VL ViT attention ( #34580 )
...
Signed-off-by: Max Hu <maxhu@nvidia.com >
Signed-off-by: Max Hu <hyoung2991@gmail.com >
Co-authored-by: Max Hu <maxhu@nvidia.com >
Co-authored-by: Shang Wang <shangw@nvidia.com >
2026-02-27 20:20:23 +08:00
Isotr0py
71cd89264f
[MM Encoder] Add Triton ViT attention backend ( #32183 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-02-15 06:32:47 -08:00
Shanshan Shen
9fb27dd3b3
[MM] Align the prefix of MMEncoderAttention with Attention ( #33750 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-02-04 04:07:30 +00:00
Shanshan Shen
5c4f2dd6ef
[MM] Pass prefix parameter to MMEncoderAttention ( #33674 )
...
Signed-off-by: shen-shanshan <467638484@qq.com >
2026-02-03 06:47:41 -08:00
Matthew Bonanni
a608b4c6c2
[5/N][Attention] Finish eliminating vllm/attention folder ( #32064 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-27 10:02:51 -05:00
Cyrus Leung
dcd80206b7
[Chore] Update type annotation of input_ids in model forward ( #33063 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-01-26 06:02:10 -08:00
Isotr0py
9ad7f89f55
[Models]: Make Multimodal config implicit in ViT implementation ( #31972 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-01-24 20:34:26 +08:00
JJJYmmm
04a9e064db
[Bugfix] fix the ima issue of qwen-vit ( #32687 )
...
Signed-off-by: JJJYmmm <92386084+JJJYmmm@users.noreply.github.com >
2026-01-20 17:21:25 +00:00
Cyrus Leung
4753f3bf69
[Model] Use context managers for encoder- and LM-only mode ( #32605 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-01-20 11:43:38 +08:00
YunzhuLu and Isotr0py
542a4059b2
[Model] Use mm_position to compute mrope positions for Qwen2-VL/2.5-VL ( #32126 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-01-13 09:04:29 +00:00
Matthew Bonanni
2612ba9285
[1/N][Attention] Restructure attention: move files ( #31916 )
...
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
2026-01-09 13:10:24 -08:00
Isotr0py
2972a05473
[MM Encoder]: Make MMEncoderAttention's scale takes effect properly ( #31950 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-01-08 02:33:48 -08:00
ce1eafd1a5
[Core] Initialize LoRA support for tower and connector in multi-modal models ( #26674 )
...
Signed-off-by: bk-201 <joy25810@foxmail.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com >
Co-authored-by: bk-201 <joy25810@foxmail.com >
Co-authored-by: prashanth058 <prashanth.dannamaneni@uipath.com >
Co-authored-by: Anexdeus <5142168@mail.ru >
2025-12-26 04:48:20 -08:00