MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
e319f86f15
[Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860 ) ( #56984 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-03 14:17:09 +00:00
Shijie Lyu
9aaef4296f
[Bugfix][Frontend] Check reused prompt token ids against the vocab before streaming ( #59555 )
...
Signed-off-by: Shijie Lyu <lshjhf@gmail.com >
2026-10-01 09:26:55 -07:00
Mia Ojeda
e006d761a5
[Bugfix][CPU][DiffusionGemma] Support narrower canvas w/ sync scheduling ( #59107 )
...
Signed-off-by: ojeda-e <estefania.b.ojeda@gmail.com >
2026-09-29 23:28:50 -04:00
b721a4c709
[Feature] Add fixed-token prefill scoring ( #54335 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-28 07:50:09 -07:00
Taneem Ibrahim
bbd7c24d6e
[MRV2] Validate MRV2 entrypoint logits processors ( #57728 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-09-24 09:07:26 +00:00
1b3b88ec2b
[Core] structured generation mode for DiffusionGemma model (Jev-like) ( #57250 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Razorback16 <razorback16@protonmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Razorback16 <razorback16@protonmail.com >
2026-09-22 20:51:17 +00:00
shaohuaxi and Codex
5abf85d27a
[Bugfix][Frontend] Validate mixed prompt embedding mask lengths ( #57006 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-22 05:50:31 +00:00
Juan Pérez de Algaba
4800024b98
[Security] Reject min_tokens that exceeds the filled max_tokens default ( #57731 )
...
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
2026-09-22 05:50:23 +00:00
Karen Chung
986e217158
[Feature] Add first-class KV hints request envelope for programmatic KV management ( #53423 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-09-21 09:56:11 -07:00
Cyrus Leung
c8824a1e42
[MM] Move supports_multimodal_inputs and cache out of registry ( #57913 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-09-21 07:43:07 -07:00
5302d1fe40
[Model Runner V2] Support custom logits processors ( #56497 )
...
Signed-off-by: David Cheung <d7cheung@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-09-19 17:46:49 +00:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
84e5f3edac
[Bugfix] Validate routed-expert prompt offsets before engine submission ( #56844 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Bhushan Asati <bhushanasati25@gmail.com >
Co-authored-by: Megha Agarwal <19240983+meghaagr13@users.noreply.github.com >
2026-09-15 09:17:41 +08:00
Juan Pérez de Algaba
e00a034ad1
(security) fix: enforce decoder prompt-length validation for skip-che… ( #46588 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-08-21 09:07:19 -07:00
Cyrus Leung
4666a8ba9e
[Refactor] Remove InputPreprocessor ( #53064 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-20 08:39:48 +00:00
a1c5b1fd9f
[Core][V1] Support trace_decode_token_ids for deterministic decode replay ( #46701 )
...
Signed-off-by: ianzxliu <ianzxliu@tencent.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com >
Co-authored-by: ianzxliu <ianzxliu@tencent.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-20 15:12:43 +08:00
b216db3ed0
[Bugfix] Reject negative token ids as out-of-vocabulary ( #51795 )
...
Signed-off-by: Junhao Shen <junshen@nvidia.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-08-14 01:07:29 +00:00
50ba4bc6b2
[Feature] Mask Replay ( #49577 )
...
Signed-off-by: vx120 <893600387@qq.com >
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-13 08:44:31 +00:00
Chauncey and Nick Hill
72c0d67657
[Feat] Support thinking_token_budget in Model Runner V2 ( #46727 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-07 03:36:18 +00:00
Karen Chung
f57123aa2d
feat(frontend): session id plumbing into requests ( #48048 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-08-03 22:18:58 +00:00
Almog Tavor
39f55ffdaa
[Core] Offload raw-prompt preprocessing to renderer thread pool in AsyncLLM ( #49608 )
...
Signed-off-by: almogtavor <almogtavor@gmail.com >
2026-08-01 15:10:51 +00:00
Zach Zhu
58f9659397
[Frontend][Core] Standardize request error handling with VLLMError hierarchy ( #49665 )
...
Signed-off-by: Zach Zhu <zzqshu@126.com >
2026-07-29 04:20:40 +00:00
Nick Hill
626fa9bba5
[BugFix] Fix blocked reasoning parsing with MRV2 ( #43808 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-05-28 04:59:34 +00:00
Luis 🚀 and Andrew Sansom
14043dfecd
feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API ( #40720 )
...
Signed-off-by: Luis Robaina <luis@protopia.ai >
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com >
Signed-off-by: LuisRobaina <luis@protopia.ai >
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com >
2026-05-01 10:05:55 +08:00
Kris Hung and Claude Opus 4.6
766cb65d00
feat(multimodal): support externally processed mm_kwargs with cache injection ( #39502 )
...
Signed-off-by: Krish Hung <krishung5@gmail.com >
Signed-off-by: krishung5 <krish@nvidia.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-04-21 11:31:09 +00:00
Chauncey
ecbfbb8d61
[Feature] Add auto-detection for reasoning_config when only reasoning_parser is set ( #38214 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-04-10 01:36:26 +00:00
Cyrus Leung
54d244f28f
[UX] Improve error message for MM input too long ( #39409 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-04-09 13:20:19 +00:00
Cyrus Leung
ba2f0acc2d
[Misc] Reorganize inputs ( #35182 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-25 10:22:54 -07:00
4731884796
[Feature] limit thinking tokens (hard limit) ( #20859 )
...
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com >
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com >
Signed-off-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-24 09:53:07 -07:00
Cyrus Leung
f96c3ab08c
[Deprecation][1/2] Remove items deprecated in v0.18 ( #36470 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-03-09 03:43:23 -07:00
Jiayi Yan and Harry Mellor
6a895197fa
[Bugfix][CI] fix typos ( #34934 )
...
Signed-off-by: 1195343015 <1195343015@qq.com >
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-05 17:05:46 +00:00
Wentao Ye and Cyrus Leung
e113a30113
[Deprecation] Deprecate code in 0.17 as scheduled ( #35441 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-02-28 17:32:37 +00:00
Cyrus Leung
761e63e541
[Frontend] Always pass supported_tasks to validation ( #35186 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-24 04:16:33 -08:00
Cyrus Leung
a766b30349
[Renderer] Deprecate code paths for old input processing ( #34775 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-18 00:35:04 -08:00
Cyrus Leung
574fe75245
[Renderer] Move InputPreprocessor into Renderer (2/2) ( #34560 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-17 05:29:01 -08:00
Seiji Eicher
79c7e09235
[KV Connector] Add temporary, off-by-default VLLM_DISABLE_REQUEST_ID_RANDOMIZATION workaround ( #34415 )
...
Signed-off-by: Seiji Eicher <seiji@anyscale.com >
2026-02-14 23:26:10 -08:00
Cyrus Leung
73391a1baa
[Renderer] Move InputPreprocessor into Renderer (1/2) ( #34510 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-02-14 10:14:21 -08:00
Cyrus Leung
2f308214c0
[Refactor] Pass full VllmConfig to Renderer ( #34485 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-12 22:48:38 -08:00
Cyrus Leung
ea5ff3a1f6
[Refactor] Simplify BOS/EOS token handling ( #34435 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-12 18:18:24 -08:00
Cyrus Leung
b96f7314b4
[Refactor] Pass Renderer to Input Processor ( #34329 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-11 19:38:11 -08:00
Cyrus Leung
ced2a92f40
[Refactor] Move validation to params definitions ( #34362 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-11 19:33:15 -08:00
Cyrus Leung
b5dcb372e4
[Misc] Clean up validation logic in input processor ( #34144 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-10 19:29:29 -08:00
Reagan Lee and Reagan Lee
7c233dbb36
[Tiny] Rename encoder budget file to more specific name ( #34103 )
...
Signed-off-by: Reagan Lee <“reaganjlee@gmail.com ”>
Co-authored-by: Reagan Lee <“reaganjlee@gmail.com ”>
2026-02-09 03:48:19 +00:00
Cyrus Leung
48312e579a
[Misc] Make PlaceholderRange.get_num_embeds a method ( #34035 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-07 05:30:17 +00:00
Cyrus Leung
cd8b405bd0
[Refactor] Consolidate sequence normalization and enc-dec parsing ( #33928 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-06 15:43:47 +00:00
Cyrus Leung and wang.yuqi
038914b7c8
[Refactor] Move task outside of PoolingParams.verify ( #33796 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-02-05 09:33:11 +00:00
Cyrus Leung
d7e17aaacd
[Refactor] Move profiling methods to MM budget ( #33559 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-02-02 23:27:00 +08:00
Cyrus Leung and YunzhuLu
21997f45b1
[Redo] #33110 with threading limit ( #33502 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
Co-authored-by: YunzhuLu <lucia.yunzhu@gmail.com >
2026-02-01 09:18:11 +00:00