MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
153a17166e
[Frontend] Add the Beam reasoning and tool parsers ( #62052 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 14:08:29 -07:00
Martin Hickey
252be28c18
[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests ( #61427 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-15 12:41:46 +00:00
Flora Feng and Chauncey
80dd705c1d
[Frontend] Make finish_reason and Responses item status truthful ( #59324 )
...
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-15 10:12:02 +01:00
Bugen Zhao and Claude Opus 5.5
8fa5ae844a
[Frontend] Let an unset --tool-strict-level fall back to the tool parser's default ( #61328 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:48:38 +08:00
Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
Kay Yan and Codex
c41b2639e2
[Bugfix][Responses API] Complete cache-write token accounting ( #60973 )
...
Signed-off-by: Kay Yan <kay.yan@daocloud.io >
Co-authored-by: Codex <noreply@openai.com >
2026-10-10 09:08:36 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
Maroon Ayoub and Claude
8a03ccf7b3
[Frontend] Add return_mm_kwargs to render requests ( #60728 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 08:14:13 +00:00
sungbin1015 and Flora Feng
2ddde19e8e
[Frontend] Support min_p in the Responses API ( #48084 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 20:38:17 +00:00
Yeonwoo Sung and Flora Feng
56e6da2c47
[Bugfix][Frontend] Honor stop_token_ids on the Responses API ( #56592 )
...
Signed-off-by: YeonwooSung <neos960518@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 18:47:30 +00:00
YueshenZ and sfeng33
ed825aafc0
[Bugfix][Frontend] Fix Responses namespace aliases in named tool calling ( #59723 )
...
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 00:07:52 +00:00
Ting SUN and Flora Feng
4a705c7b54
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null ( #47838 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:43:31 +00:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
Yufeng He and Flora Feng
6f75cc7d5d
[Bugfix][Responses API] Reuse streamed item ids in final harmony response ( #59859 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-04 03:01:04 +00:00
9bd993564f
[Frontend] Parse tool calls and reasoning from checkpoint response templates ( #58604 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 02:33:00 +00:00
2a52715ef7
[Bugfix][Responses] Use standard reasoning content-part events ( #59652 )
...
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-01 19:20:57 +00:00
Juan Pérez de Algaba and Cursor
cff08b461e
[Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… ( #50300 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-30 08:32:00 -07:00
Flora Feng
7ca31ef599
[Bugfix][Responses API] Build streamed final response from streamed items ( #59307 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-30 00:16:05 +00:00
RongJie G and Flora Feng
0755e69e75
[Bugfix][Responses API] Preserve built-in tool output call IDs ( #55596 )
...
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-29 22:45:14 +00:00
Flora Feng
797505747f
[Bugfix][Frontend] Honor parallel_tool_calls=false in the Responses API ( #59298 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-29 22:36:11 +00:00
Flora Feng
3184226984
[Bugfix][Frontend] Count Responses reasoning tokens per tool round ( #58927 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 00:34:38 +00:00
yzong-rh and Ben Browning
b6761e8ded
[Bugfix] Default missing detail for Responses API input images ( #57241 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
2026-09-25 14:08:55 -07:00
7871963fcc
[Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop ( #58551 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 14:00:39 -07:00
dee37d8911
[Frontend] Add --tool-strict-level for server-side control for structural tag activation ( #56268 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: lazymio <mio@lazym.io >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-18 04:32:39 +00:00
xinnywinne and Benjamin Chislett
fbf2c5e8be
[Frontend] Add per-request metrics to Responses API ( #55084 )
...
Signed-off-by: wxin <wxin@nvidia.com >
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com >
2026-09-16 22:52:05 +00:00
yzong-rh and Flora Feng
5a6ccc5892
[Bugfix] Add Responses cache_write_tokens for API compat and CC parity ( #57222 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-16 21:52:34 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
shaohuaxi and Codex
e0301afb2a
[Bugfix][Responses] Clean up MCP tool sessions once before closing them ( #56988 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-16 14:07:06 +08:00
Nicole LiHui 🥜 and Chauncey
7ef4d9bfed
[Bugfix][Responses] Fix browser.find action type ( #55305 )
...
Signed-off-by: nicole-lihui <nicole.li@daocloud.io >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-11 12:07:04 +00:00
ea40bb9e90
[feature] Watermarked generation and detection (Gumbel-max algorithm) ( #54053 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-09 21:00:15 -07:00
Nikhil Kulkarni and Claude Fable 5.1
114abd1c11
[Bugfix][Frontend] Reject unsupported Responses API input items with 400 instead of 500 ( #55974 )
...
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 07:51:45 +00:00
Adababy and shaolila
167858ea17
[Frontend] Migrate Responses harmony input validation to VLLMValidationError ( #55701 )
...
Signed-off-by: shaolila <shaolila@buaa.edu.cn >
Co-authored-by: shaolila <shaolila@buaa.edu.cn >
2026-09-07 13:30:11 +00:00
Francisco Javier Arceo and OpenAI Codex
6a2a2bb02b
[Frontend] Add stateless /v1/responses/render endpoint ( #50195 )
...
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-07 12:50:42 +00:00
Adababy and shaolila
7dbe386833
[Frontend] Migrate Responses API validation errors to VLLMValidationError ( #50257 )
...
Signed-off-by: shaolila <shaolila@buaa.edu.cn >
Co-authored-by: shaolila <shaolila@buaa.edu.cn >
2026-09-07 17:33:51 +08:00
e962733e08
[Security] Validate cache salts before they reach LMCache ( #51444 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-05 03:56:06 +00:00
Nicolò Lucchesi
dafbef15a1
[Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management ( #49445 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-31 14:26:49 +00:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
Juan Pérez de Algaba
1dc464d426
[Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion ( #54353 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-08-30 04:28:16 +00:00
Taneem Ibrahim
d7b6a35967
[Bugfix] Handle malformed namespace tools ( #53763 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-27 14:18:16 -04:00
wang.yuqi
eac636a7fa
[Frontend] Move api_server.py out openai folder ( #52131 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-19 09:43:46 +00:00
Kaif Kohari
a6a2a93f9b
[Bugfix][Frontend] Guard remaining before-validators against non-object JSON bodies ( #52528 )
...
Signed-off-by: Kaif <kaifkohari10@gmail.com >
2026-08-17 02:27:59 +00:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ac2ae8798c
[Frontend] Support count_reasoning_tokens in the Streaming Parser Engine ( #45802 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-15 06:51:14 -07:00
b64a2708b0
Bound generation inputs before expensive work ( #51447 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 22:03:50 +08:00
Chauncey
22170354d8
[Feature] Parse request priority from HTTP header ( #51089 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-05 21:10:29 -07:00
Karen Chung
f57123aa2d
feat(frontend): session id plumbing into requests ( #48048 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-08-03 22:18:58 +00:00
Cyrus Leung
b3f97dae24
[Frontend] Require cache_salt to be non-empty via schema ( #50816 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-03 06:51:27 +00:00
yzong-rh
dc818c198d
[GPT-OSS] Strict tool call and constrained decoding for Harmony ( #45560 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-08-01 12:10:26 -04:00
yzong-rh and Ben Browning
e8b358b4c0
[Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity ( #50334 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
2026-07-31 15:45:40 -04:00