MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
Bruno Volpato and Codex
61f084b5f3
[Bugfix][Multimodal] Fix per-video cache option reuse ( #55203 )
...
Signed-off-by: bvolpato <brunocvcunha@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-11 06:21:55 +00:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
tenotek and Claude Fable 5.1
443dccd3c9
[Bugfix][Multimodal] Fix reference counting of the SHM processor cache ( #59695 )
...
Signed-off-by: tenotek <50898317+hdq66666@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-02 22:14:13 +08:00
Daniel M. García-Ocaña Hernández
295a0b8d64
[Bugfix][Multimodal] Fix flat/scoped mm_processor_kwargs merge and resolution ( #56372 )
...
Signed-off-by: Daniel M. García-Ocaña Hernández <danielgarciaocana@gmail.com >
2026-09-29 03:58:39 +00:00
Harry Mellor and Claude Opus 5.5
d881812500
[Chore] Use Transformers v5 names and drop redundant processor use_fast ( #58550 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-24 14:14:10 +00:00
Cyrus Leung
7853700d4c
[MM] Move get_dummy_processor_inputs into MM processor ( #57967 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 09:39:20 -07:00
Cyrus Leung
751f6807d9
[Refactor] Use dynamic MM cache in processor ( #57674 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-19 15:38:00 +00:00
Harry Mellor and Claude Opus 5
8a5cf54387
[Multimodal] Type dummy options per modality ( #57576 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-18 17:29:11 +01:00
Cyrus Leung
0b9e018fb9
[MM] Further cleanup _apply_hf_processor_main ( #53610 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-16 10:21:00 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Hung Hoang (黄兴) and Cursor Agent
5ff50f3996
[Bugfix][MM] Fix swapped H/W in dummy video profiling inputs ( #56385 )
...
Signed-off-by: hungh <hungh@nvidia.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-09-11 05:25:51 +00:00
Luyi Xiao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8a728663c1
[Bugfix] Reject tokenizer-less Qwen VL processor init ( #54886 )
...
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-03 21:25:35 -07:00
waizuichougou
c6bca6e585
[Bugfix][Multimodal] Scope cache hash kwargs by modality ( #54918 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
2026-09-02 14:20:26 +00:00
Jan Kwieciński
1f1f628859
[Feat][MM Hashing] include media_io_kwargs in multi-modal hashes ( #54241 )
...
Signed-off-by: Jan Kwiecinski <jkwiecinski@nvidia.com >
2026-09-01 13:17:01 +00:00
Prudhvi Vuda
7292ee2791
[Bugfix][Multimodal] Honor modality-scoped mm_processor_kwargs ( #53808 )
...
Signed-off-by: Prudhvivuda <prudhvi12042001@gmail.com >
2026-08-31 06:37:12 +00:00
waizuichougou and OpenAI Codex
f6895a5fcb
[Bugfix][Multimodal] Avoid caching full prompts in fallback ( #54439 )
...
Signed-off-by: waizuichougou <2082431897@qq.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-30 16:19:38 +00:00
Cyrus Leung
ecfa7bb373
[MM] Cache common token sequences ( #53560 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-24 07:48:10 -07:00
Cyrus Leung and Qwen Code
ebcd606467
[MM] Simplify prompt updates: replace PromptSeq with list[int] ( #53372 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: Qwen Code <noreply@alibaba-inc.com >
2026-08-23 14:50:32 +08:00
Cyrus Leung
020829f66b
[MM] Remove renderer_applies_updates flag ( #53385 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-23 01:25:55 +08:00
Cyrus Leung
040700aaa6
[MM] Address comments on #53275 ( #53364 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-22 10:27:59 +00:00
Cyrus Leung
0b19ebcacd
[MM] Simplify _apply_hf_processor_main ( #53275 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-22 01:49:37 -07:00
Cyrus Leung
b00f475f09
[MM] Remove text components from ProcessorInputs ( #53093 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-20 20:42:53 -07:00
Yiqin and shenyiqin
cb09dd7488
[Core][Multimodal] Skip redundant placeholder scan when token match succeeds ( #52925 )
...
Co-authored-by: shenyiqin <shenyiqin1@huawei.com >
2026-08-20 15:04:29 +00:00
liuzhenwei
b1d9337e92
[EPD] Allow KV consumers to omit MM embeddings ( #52697 )
...
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com >
2026-08-19 02:16:08 +00:00
Wentao Ye
9a276d6375
[Mypy Fix] Mypy fix for "vllm/model_executor/models/[cC][dD]" ( #52003 )
2026-08-13 11:33:55 +08:00
Tianyu Guo and OpenAI Codex
3fb7bb46d6
[Perf] Avoid repeated multimodal prompt update scans ( #51774 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-11 09:23:24 +00:00
Yunzhu Lu and Cyrus Leung
f84df12c6a
[Bugfix][MM] Fix MiniCPM-V placeholder replacement and image processor loading on Transformers v5 ( #48413 )
...
Signed-off-by: YunzhuLu <lucia.yunzhu@gmail.com >
Signed-off-by: Yunzhu Lu <lucia.yunzhu@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-08-05 21:12:00 -07:00
Tianyu Guo
b30291cf0c
[EPD] Remove duplicate image preprocessing in EPD and enable preprocess on GPU ( #50390 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-05 15:59:54 -07:00
Hareesh Gali and Claude Fable 5
89ac407e3d
[Perf] Speed up multimodal placeholder and token-match scanning ( #50716 )
...
Signed-off-by: Hareesh <hgsata@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-08-03 08:27:56 +00:00
Isotr0py
1180b604f8
[Multimodal] Expose mm hash algothrim selection to cli args ( #49686 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
2026-07-31 05:25:19 +00:00
Guan-Ming Chiu
aeaa50a71c
[Bugfix][Multimodal] Include media IO config in MM cache hash ( #49975 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
2026-07-29 10:48:40 +00:00
Andreas Karatzas
f0f6805d8a
[CI] Stabilize the multi-audio OpenAI server path ( #44051 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-06-07 15:54:32 +08:00
yzong-rh and wang.yuqi
20dcd984f9
[Bugfix] Fix RuntimeError: Already borrowed by adding thread-safe Hugging Face fast-tokenizer wrappers ( #41181 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-05 14:04:01 +00:00
Cyrus Leung
502c41a8f6
[Model] Use helper function to run MM processors with token inputs (where applicable) ( #38018 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-26 16:44:04 +08:00
Cyrus Leung
ba2f0acc2d
[Misc] Reorganize inputs ( #35182 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-25 10:22:54 -07:00
Ekagra Ranjan
b5ca9c3557
[Models] Cohere ASR ( #35809 )
...
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com >
2026-03-17 21:04:17 +00:00
Cyrus Leung
f96c3ab08c
[Deprecation][1/2] Remove items deprecated in v0.18 ( #36470 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-09 03:43:23 -07:00
Cyrus Leung
d62856b928
[Misc] Move processors to transformers_utils ( #35953 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-09 11:31:39 +08:00
Martin Hickey
c3598d02fa
[Misc] Remove deprecated items that are due for removal ( #36006 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-03-05 06:14:50 +00:00
Wentao Ye and Cyrus Leung
e113a30113
[Deprecation] Deprecate code in 0.17 as scheduled ( #35441 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-02-28 17:32:37 +00:00
Cyrus Leung
392645454b
[Refactor] Decouple TimingContext from InputProcessingContext ( #35083 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-23 14:15:50 +00:00
Neil Schemenauer
54e2f83d0a
[Feature] Lazy import for the "mistral" tokenizer module. ( #34651 )
...
Signed-off-by: Neil Schemenauer <nas@arctrix.com >
2026-02-23 00:43:01 -08:00
Cyrus Leung
987506bca6
[Refactor] Simplify dummy data generation ( #35025 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-22 20:55:27 -08:00
Cyrus Leung
a0d8d944e2
[Renderer] Move MM Hash parsing into Renderer ( #34711 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-17 19:18:55 -08:00
Cyrus Leung
574fe75245
[Renderer] Move InputPreprocessor into Renderer (2/2) ( #34560 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-17 05:29:01 -08:00
Cyrus Leung
ec17bdd894
[Renderer] Move InputPreprocessor into Renderer (1.5/2) ( #34598 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-15 23:46:33 -08:00
Cyrus Leung
73391a1baa
[Renderer] Move InputPreprocessor into Renderer (1/2) ( #34510 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-02-14 10:14:21 -08:00
Cyrus Leung
372b2e762a
[Bugfix] Standardize getting number of image patches/tokens ( #34358 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-12 20:47:01 -08:00
Isotr0py
0ab06100f4
[Multimodal] Expose mm_processor_kwargs for DummyInputsBuilder ( #34330 )
...
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-02-11 09:37:40 -08:00