MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
Junyi Yao and sfeng33
fdfc171a74
[Bugfix][ToolParser] Stream every tool call when one delta completes several ( #60354 )
...
Signed-off-by: Junyi Yao <j.yao@wustl.edu >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 01:07:35 +00:00
YueshenZ and sfeng33
ed825aafc0
[Bugfix][Frontend] Fix Responses namespace aliases in named tool calling ( #59723 )
...
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 00:07:52 +00:00
lzhan011 and Flora Feng
9bf4e51030
[Bugfix][Frontend] Malformed $defs in a tool schema should be a 400, not a 500 ( #54850 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 01:37:20 +00:00
RomeJay and Flora Feng
2e06395fd3
[Bugfix][Frontend] Build tool-call grammars from the tools the prompt renders ( #59879 )
...
Signed-off-by: Rongjie Jiang <j1870831863@outlook.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:01:03 +00:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
Ting SUN and Flora Feng
fcf4750466
[Bugfix][Frontend] Constrain forced named tool choice with empty parameters to a JSON object ( #45290 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-30 05:44:35 +00:00
Ostring24 and Flora Feng
4e7af3b0de
[Bugfix][Frontend] Enforce parallel_tool_calls=false in the required-tool grammar ( #50502 )
...
Signed-off-by: ostring <ostring@qq.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-29 21:33:28 +00:00
5dd693122c
[MyPy][2/N] Fix mypy errors in small tests/ dirs ( #51043 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 21:27:45 +01:00
yzong-rh and Ben Browning
b6761e8ded
[Bugfix] Default missing detail for Responses API input images ( #57241 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
2026-09-25 14:08:55 -07:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
6adad08767
Add Muse Glimmer model support ( #51655 )
...
Signed-off-by: Tiezhen Wang <tiezhen@inferact.ai >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Beto de Paola <betodepaola@meta.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: zixi-qi <zixi@inferact.ai >
2026-08-14 12:28:56 +08:00
Bugen Zhao
ab98034d4c
[Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness ( #50420 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-31 10:02:17 +08:00
Julien Denize
1a20d23dab
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls ( #48947 )
2026-07-30 15:38:00 +02:00
Bugen Zhao
f5a7cce9b6
[Model] Add Kimi K3 support: Python frontend [2/2] ( #50093 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-29 01:06:01 -07:00
Zach Zhu
58f9659397
[Frontend][Core] Standardize request error handling with VLLMError hierarchy ( #49665 )
...
Signed-off-by: Zach Zhu <zzqshu@126.com >
2026-07-29 04:20:40 +00:00
Harry Mellor and Andreas Karatzas
ca0defa343
Make bare hugging_face imports forbidden ( #49726 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-25 04:18:39 -07:00
c0302d9497
[Bugfix] Fix parallel_tool_calls=null crash in Responses API from_request() ( #48098 )
...
Signed-off-by: mahadrehmann <mahadrehman04@gmail.com >
Signed-off-by: Mahad Rehman <114791389+mahadrehmann@users.noreply.github.com >
Co-authored-by: muhammadfawaz1 <135441198+professorsab@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-07-15 18:01:17 +08:00
akii96 and Andreas Karatzas
5769a7382c
[ROCm][CI][Bugfix] Fix flaky parallel tool-call streaming (test assertion + Mistral/Granite parsers) ( #47550 )
...
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com >
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-06 19:06:19 -04:00
Ben Browning and Claude Opus 4.6
25671cb520
[Parser][Bugfix] Ensure tool call or other special tokens don't leak in non-streaming tool parsing ( #46875 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-06-30 13:46:53 -04:00
Ting SUN and Flora Feng
899d72a58c
[Bugfix][ToolParser] Handle braces in required tool streaming strings ( #45389 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-06-23 20:29:34 +00:00
Federico
b831374cf1
[Bugfix][Gemma4] Fix parsing when thinking is disabled ( #45832 )
...
Signed-off-by: Federico Iezzi <fiezzi@google.com >
2026-06-17 02:41:36 +00:00
Federico
b9684d99e9
[Bugfix] Gemma4: skip forced JSON for required/named tool choice ( #45795 )
...
Signed-off-by: Federico Iezzi <fiezzi@google.com >
2026-06-16 20:38:29 +00:00
Ben Browning and Flora Feng
76a373eff4
[Frontend] Replace legacy Gemma4 parsers with engine-based implementation ( #45588 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-06-15 21:34:07 +00:00
Ben Browning
d1bcb4b44c
[Bugfix] Fix tool parsing crash with non-function tool types (e.g. WebSearchTool) ( #45147 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
2026-06-10 17:17:16 +00:00
Umut Polat
f46e6be169
[Misc] Use VLLMValidationError consistently in chat completion and completion protocol validators ( #36254 )
...
Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com >
2026-06-01 04:04:11 +00:00
Andreas Karatzas
76ea1d5d2f
[ROCm][CI] Stabilize Granite tool-use and test URL construction ( #43017 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-23 12:21:11 +08:00
Ben Browning and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
346cf163a1
[Frontend] Normalize reasoning_content to reasoning for client compatibility ( #42664 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-05-20 21:23:47 -07:00
Flora Feng
8eb401134e
[Refactor] Consolidate required/named tool_choice streaming into DelegatingParser ( #41876 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-05-07 09:50:59 -07:00
Flora Feng and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8317cedc77
[Responses] Add tool_choice/tools validation to match OpenAI behavior ( #40399 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-04-22 22:46:10 -07:00
Hoang Nguyen and Claude
6e10cb54f6
[Bugfix][Responses API] Fix streaming tool calls on /v1/responses ( #39892 )
...
Signed-off-by: Hoang Nguyen <118159510+hnt2601@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-20 11:24:52 +08:00
jigangz and Claude
18013df6ae
[Bugfix] Reject empty tools array with HTTP 400 ( #39780 )
...
Signed-off-by: Jigang Zhou <zjg0907008@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-16 12:08:04 +08:00
Julien Denize
c0722f22de
[Mistral Grammar] Fix tool and reasoning parsing ( #39217 )
...
Signed-off-by: juliendenize <julien.denize@mistral.ai >
2026-04-15 21:05:04 -07:00
Flora Feng
f04d5226f8
[CI] Fix flaky tool_use chat completion tests with deterministic seed ( #37027 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-03-17 03:24:34 +00:00
Flora Feng
8647c6cf51
[Bugfix] Fix minimax_m2 tool parser when stream interval > 1 ( #35895 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-03-12 10:25:14 +08:00
Chauncey
fefce49807
[Refactor] [6/N] to simplify the vLLM openai chat_completion serving architecture ( #32240 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-01-13 13:01:39 +00:00
Nick Hill
c6bb5b5603
[BugFix] Fix engine crash caused by chat tools + response_format ( #32127 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-01-13 10:33:14 +08:00
Mary and Chauncey
7645bc524b
[OpenAI] Fix tool_choice=required streaming when output has trailing extra data ( #31610 )
...
Signed-off-by: maylikenoother <ogedengbemary19@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-01-08 21:01:42 +08:00
Robin
20fda43151
[Bugfix][Frontend] Prevent IndexError in MiniMax M2 tool parser during streaming extraction ( #30555 )
...
Signed-off-by: WangErXiao <863579016@qq.com >
2025-12-17 16:37:57 +08:00
Cyrus Leung
ed586e7724
[Refactor] [3/N] Move tool parser tests and run on CPU ( #30693 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-15 13:45:36 +00:00
Chauncey
2a1776b7ac
[Refactor] [2/N] Move tool parsers into the vLLM main directory ( #30675 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2025-12-15 12:54:52 +00:00
Cyrus Leung
64251f48df
[Chore] Adjust tokenizer import to avoid circular imports ( #30601 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-13 04:42:39 -08:00
Julien Denize
5c213d2899
[BUGFIX] Mistral tool call parser v11+ ( #30332 )
...
Signed-off-by: juliendenize <julien.denize@mistral.ai >
2025-12-09 14:55:38 +00:00
dd5d1ef780
[Bugfix] Mistral tool parser streaming update ( #19425 )
...
Signed-off-by: avigny <47987522+avigny@users.noreply.github.com >
Signed-off-by: Chauncey <chaunceyjiang@gmail.com >
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: Jeff Cook <jeff@jeffcook.io >
Co-authored-by: sfbemerk <benjaminmerkel@mail.de >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2025-12-03 17:45:31 +00:00
Cyrus Leung
653591d5e7
[Chore] Move tokenizer initialization methods ( #29793 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-12-02 13:33:37 +08:00
Cyrus Leung
fa59fe417f
[Chore] Move detokenizer_utils to vllm/tokenizers ( #29727 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2025-11-29 06:25:17 -08:00