MOVZX and Hinata
5aa2ac64f8
[Mini-VLLM] Remove LoRA, plugins, speech-to-text and scale-out surfaces
...
Production runs one static model behind the OpenAI-compatible API. Every
adapter path and non-OpenAI HTTP surface is unreachable there.
- Delete the LoRA stack: vllm/lora, LoRARequest on the engine wire, the
scheduler, worker, cudagraph and MoE plumbing, LoRA metrics, the lora
CLI group, and the LoRA tests.
- Drop the plugin loader in vllm/plugins. Platform resolution is now
CUDA-only and the general/stat-logger plugin hooks are gone.
- Delete the speech-to-text and realtime entrypoints, the scale-out
render routes, and vllm/config/speech_to_text.py. GenerationTask is
generate-only.
- Cut DecodeBenchConnector and the disagg-prefill doc and image. The
Multi, Offloading and Example connectors stay; production uses the
CPU KV offload tier.
- Delete the ROCm-only fused KDA chunk kernel (2346 lines) with its CMake
arch gate, declarations and op registrations. The CUDA build never
defined the guard and no Python caller existed.
- Delete the orphan Mistral and Pixtral chat fixtures and examples/.
- Fix _check_model: removing the LoRA early return made every request
fail with "The model ... does not exist. Valid aliases: ...".
Verified on an RTX 3090: import sweep reports 1242 modules with 7 known
optional-dependency failures; a live server answers chat, completions,
responses, tokenize and detokenize with zero log errors and 404 for the
removed surfaces; collection yields 8558 tests with the 3 pre-existing
errors; ruff adds no new finding; the incremental _C build succeeds.
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 21:06:19 +07:00
MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
Ting SUN
eb8de8d1a1
[Bugfix][OpenAI] Respect allowed_token_ids in beam search ( #47630 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-17 08:50:40 +00:00
Jovan and OpenAI Codex
5be4266f5c
[Bugfix][Frontend] Preserve Cohere and Anthropic streaming errors ( #61947 )
...
Signed-off-by: Kevin <jia__kuan@163.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-17 06:10:16 +00:00
Shuolei Wang and aoshen02
b3a7e2ff6e
[RL][Frontend] Return weight-version spans on OpenAI completions and chat ( #61347 )
...
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-17 02:08:07 +00:00
153a17166e
[Frontend] Add the Beam reasoning and tool parsers ( #62052 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 14:08:29 -07:00
3dcfaeb4e8
[Frontend] Apply each endpoint's top_logprobs cut in derender ( #59518 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-15 15:04:04 +00:00
Martin Hickey
252be28c18
[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests ( #61427 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-15 12:41:46 +00:00
Flora Feng and Chauncey
80dd705c1d
[Frontend] Make finish_reason and Responses item status truthful ( #59324 )
...
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-15 10:12:02 +01:00
Bugen Zhao and Claude Opus 5.5
8fa5ae844a
[Frontend] Let an unset --tool-strict-level fall back to the tool parser's default ( #61328 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:48:38 +08:00
Xunzhuo
89cab55086
[Model] Support Decision 3.0 decision models (vllm-sr/d3) ( #61565 )
...
Signed-off-by: Xunzhuo Liu <xunzhuo.liu@amd.com >
2026-10-14 17:10:55 +00:00
Chauncey and Cyrus Leung
e1726d9142
[Frontend] Add cache salt and tracing to openai decisions ( #61550 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-14 00:48:44 -07:00
c923e7769b
[Feat] Add text-only OpenAI Decisions API MVP ( #60465 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-14 07:51:08 +08:00
GuanLuo and sfeng33
6d1daa9fe5
[Bugfix][Frontend] Expose completion response schemas in OpenAPI ( #60926 )
...
Signed-off-by: Guan Luo <gluo@nvidia.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-13 18:54:06 +00:00
Chester
c599b5971f
[Bugfix][Frontend] Return all logprobs when completions requests logprobs=-1 ( #55717 )
...
Signed-off-by: Chester <42577861+superdiaodiao@users.noreply.github.com >
2026-10-12 17:05:55 +00:00
Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
Kay Yan and Codex
c41b2639e2
[Bugfix][Responses API] Complete cache-write token accounting ( #60973 )
...
Signed-off-by: Kay Yan <kay.yan@daocloud.io >
Co-authored-by: Codex <noreply@openai.com >
2026-10-10 09:08:36 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
premsurawut and Claude Opus 5.5
c811943137
[Bugfix][Frontend] Reject beam search with echo and logprobs in /v1/completions ( #60876 )
...
Signed-off-by: surawut.j <surawut.jirasaktavee@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 18:06:26 +00:00
Maroon Ayoub and Claude
8a03ccf7b3
[Frontend] Add return_mm_kwargs to render requests ( #60728 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-09 08:14:13 +00:00
Vineeth Sai Varikuntla and Flora Feng
112a282915
[Bugfix][Frontend] Keep streamed completion text_offset in step with tokens ( #60668 )
...
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 21:20:43 -07:00
sungbin1015 and Flora Feng
2ddde19e8e
[Frontend] Support min_p in the Responses API ( #48084 )
...
Signed-off-by: sungbin1015 <sbin@solbox.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 20:38:17 +00:00
dingdangmao and Flora Feng
bab0ac61f8
[Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions ( #59263 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 19:32:46 +00:00
Yeonwoo Sung and Flora Feng
56e6da2c47
[Bugfix][Frontend] Honor stop_token_ids on the Responses API ( #56592 )
...
Signed-off-by: YeonwooSung <neos960518@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 18:47:30 +00:00
YueshenZ and sfeng33
ed825aafc0
[Bugfix][Frontend] Fix Responses namespace aliases in named tool calling ( #59723 )
...
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 00:07:52 +00:00
Sriraj and Flora Feng
d77cf81156
[Bugfix][Frontend] Honor default penalties in batch chat completions ( #60307 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:21:00 +00:00
Ting SUN and Flora Feng
4a705c7b54
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null ( #47838 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:43:31 +00:00
Yida Weng and Flora Feng
18c8a65edd
[BugFix][Frontend] Pass reasoning_ended through render → generate ( #60059 ) ( #60062 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 20:39:25 +00:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
Stefan Wang and Robert Shaw
b0eb87fe49
[Bugfix][Frontend] Return streaming errors before the first token ( #40986 )
...
Signed-off-by: 1fanwang <1fannnw@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-04 16:42:49 +00:00
Yufeng He and Flora Feng
6f75cc7d5d
[Bugfix][Responses API] Reuse streamed item ids in final harmony response ( #59859 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-04 03:01:04 +00:00
9bd993564f
[Frontend] Parse tool calls and reasoning from checkpoint response templates ( #58604 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 02:33:00 +00:00
2a52715ef7
[Bugfix][Responses] Use standard reasoning content-part events ( #59652 )
...
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-01 19:20:57 +00:00
Nick Hill and Claude Opus 5.5
c055c1e075
[Core] Include the LoRA path in prefix-cache block hashes ( #59335 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 08:37:09 +08:00
Juan Pérez de Algaba and Cursor
cff08b461e
[Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… ( #50300 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-30 08:32:00 -07:00
Shijie Lyu
ac68c30872
[Bugfix][Frontend] Ignore reused prompt token ids for media in /v1/responses ( #59173 )
...
Signed-off-by: Shijie Lyu <lshjhf@gmail.com >
2026-09-30 05:21:09 +00:00
Nick Hill and Claude Opus 5.5
105a4e097b
[Bugfix][Frontend] Reject LoRA adapters named after a served model ( #59286 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 18:29:38 -07:00
Flora Feng
7ca31ef599
[Bugfix][Responses API] Build streamed final response from streamed items ( #59307 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-30 00:16:05 +00:00
RongJie G and Flora Feng
0755e69e75
[Bugfix][Responses API] Preserve built-in tool output call IDs ( #55596 )
...
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-29 22:45:14 +00:00
Flora Feng
797505747f
[Bugfix][Frontend] Honor parallel_tool_calls=false in the Responses API ( #59298 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-29 22:36:11 +00:00
c6843d397c
[Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo ( #55771 )
...
Signed-off-by: Shijie Lyu <sjlyu@amazon.com >
Co-authored-by: Shijie Lyu <sjlyu@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-29 10:36:22 +08:00
Flora Feng
6f65fde419
[Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions ( #58958 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 04:12:04 +00:00
Flora Feng
027b6f3a22
[Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions ( #58939 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 02:01:35 +00:00
Flora Feng
4d07e90653
[Bugfix][Frontend] Sample batched chat completions from the adjusted requests ( #58929 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 01:46:55 +00:00
Flora Feng
3184226984
[Bugfix][Frontend] Count Responses reasoning tokens per tool round ( #58927 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 00:34:38 +00:00
73859fec58
[Frontend] Switch Python Harmony dependency to oss-harmony ( #55128 )
...
Signed-off-by: Anton Peganov <apeganov@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 00:43:56 +04:00
yzong-rh and Ben Browning
b6761e8ded
[Bugfix] Default missing detail for Responses API input images ( #57241 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
2026-09-25 14:08:55 -07:00
8b47e8b22c
[Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams ( #56067 )
...
Signed-off-by: Cheng Rui <286040359@qq.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 14:08:23 -07:00
7871963fcc
[Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop ( #58551 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-25 14:00:39 -07:00
f339e0e757
[Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks ( #58583 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com >
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 16:33:19 +00:00