Yeonwoo Sung and Flora Feng
56e6da2c47
[Bugfix][Frontend] Honor stop_token_ids on the Responses API ( #56592 )
...
Signed-off-by: YeonwooSung <neos960518@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 18:47:30 +00:00
Francisco Javier Arceo and Misha Goin
705e124c05
[Bugfix] Followup to #59299 : preserve Unicode in structured decision state prompts ( #60651 )
...
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
2026-10-08 11:00:03 -07:00
d33872dfeb
[Feat] Structured decisions endpoint (/v1/systemone) ( #59299 )
...
Signed-off-by: Matt Mastracci <matthew@mastracci.com >
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com >
Co-authored-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-08 14:48:12 +00:00
wang.yuqi
459764104d
[Frontend] Move offline_utils.py to entrypoints/common. ( #58052 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-08 09:56:05 +00:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
Ting SUN
0f112d1800
[Bugfix][Pooling] Preserve cache_salt for chunked embeddings ( #47696 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-10-08 07:33:28 +00:00
f226af0e11
[Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners ( #54113 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 06:25:27 +00:00
4187ade7b8
[Frontend] Consolidate rl entrypoints. ( #57849 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 21:37:11 -07:00
bnkaiteam and Flora Feng
3722645539
[Bugfix][Frontend] Accept successful 2xx responses for batch output uploads ( #60507 )
...
Signed-off-by: bnkaiteam <bnkaiteam@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 02:32:35 +00:00
2dd887426e
[Bugfix][Frontend] Do not modify caller params in LLM.score() ( #59959 )
...
Signed-off-by: 叮当猫 <2756823972@qq.com >
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: DeepSeek Harness <noreply@deepseek.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
2026-10-08 09:03:08 +08:00
Kang Nahoon and Flora Feng
1995c0fcd0
[Bugfix][Frontend] Apply output_config and thinking in Anthropic count_tokens ( #59180 )
...
Signed-off-by: Kang Nahoon <ejrtks102020@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-08 00:24:20 +00:00
YueshenZ and sfeng33
ed825aafc0
[Bugfix][Frontend] Fix Responses namespace aliases in named tool calling ( #59723 )
...
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 00:07:52 +00:00
Sriraj and Claude
43b4aaea3e
[Bugfix][Frontend] Return 4xx for client errors in /cohere/v2/chat ( #60309 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 04:55:34 +00:00
Mohammad Hijjawi and Cyrus Leung
ef014bdf06
[Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses ( #59072 )
...
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-07 04:13:52 +00:00
Sriraj and Flora Feng
db6e3cd8c4
[Bugfix][Frontend] Log the full request body when max_log_len is unset ( #60310 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:42:17 +00:00
Sriraj and Flora Feng
d77cf81156
[Bugfix][Frontend] Honor default penalties in batch chat completions ( #60307 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:21:00 +00:00
7b665f7a58
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) ( #58181 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 17:22:47 +00:00
09e0ce3d82
[Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server ( #54835 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-06 18:46:24 +08:00
czhu-cohere and aoshen02
6fddc9b21e
[Profiler] Add per-session profiling controls to Python and Rust frontends ( #57875 )
...
Signed-off-by: czhu <conway.zhu@cohere.com >
Signed-off-by: Conway Zhu <conway.zhu@cohere.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-06 05:21:19 +00:00
Mai Hoàng Đức and Flora Feng
5cd8a2626b
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls ( #60036 )
...
Signed-off-by: Mazukiri <maihoangduc04@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 00:06:59 +00:00
Ting SUN and Flora Feng
4a705c7b54
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null ( #47838 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:43:31 +00:00
Yida Weng and Flora Feng
18c8a65edd
[BugFix][Frontend] Pass reasoning_ended through render → generate ( #60059 ) ( #60062 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 20:39:25 +00:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
Tyrone and Harry Mellor
7867d6c52d
[Frontend] Name the served models in the model-not-found 404 ( #59889 )
...
Signed-off-by: TyroneNel <71038642+TyroneNel@users.noreply.github.com >
Signed-off-by: TyroneNel <Tyrone.Nel@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-04 20:45:51 +00:00
Stefan Wang and Robert Shaw
b0eb87fe49
[Bugfix][Frontend] Return streaming errors before the first token ( #40986 )
...
Signed-off-by: 1fanwang <1fannnw@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-04 16:42:49 +00:00
Yufeng He and Flora Feng
6f75cc7d5d
[Bugfix][Responses API] Reuse streamed item ids in final harmony response ( #59859 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-04 03:01:04 +00:00
Ting SUN and Flora Feng
50b404e71e
[Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams ( #47933 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-04 02:57:26 +00:00
Martin Hickey
bc21cba967
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) ( #58588 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-03 19:38:25 +08:00
592c6f3fbc
[Bugfix][Pooling] Handle multimodal cache misses without crashing ( #58975 )
...
Signed-off-by: linchaoying6 <linchaoying6@jd.com >
Co-authored-by: linchaoying6 <linchaoying6@jd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-02 10:14:33 -04:00
9d2a6f52b2
[Rust Frontend] Expose gRPC port in Python vllm serve ( #59659 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 08:14:37 +00:00
Martin Hickey
fcfacc3499
[Bugfix][Frontend] Seed derender detokenization from the prompt ( #59046 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-02 03:17:56 +00:00
9bd993564f
[Frontend] Parse tool calls and reasoning from checkpoint response templates ( #58604 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 02:33:00 +00:00
28c57456db
[Bugfix][Frontend] Accept Anthropic tool_addition and tool_removal content blocks ( #57693 )
...
Signed-off-by: Anurag Singh <dr.anuragkr@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Antigravity <antigravity@google.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-01 20:03:36 -04:00
2a52715ef7
[Bugfix][Responses] Use standard reasoning content-part events ( #59652 )
...
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-01 19:20:57 +00:00
e9be5323f0
[Perf] Integrate flash-maxsim Triton kernels for late-interaction scoring ( #40337 )
...
Signed-off-by: roi.pony <roi.pony@ibm.com >
Co-authored-by: roi.pony <roi.pony@ibm.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-10-01 14:06:43 -04:00
5d9214d6b2
[Bugfix][Frontend] Strip x-anthropic-billing-header billing header from /v1/chatcompletions ( #59419 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-01 13:44:40 +00:00
Mark McLoughlin and OpenAI Codex
91fcec1235
[Core][Logging] Add built-in JSON formatter ( #58739 )
...
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-01 09:11:46 +01:00
4a9d6f9572
[Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs ( #55935 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-01 14:43:38 +08:00
Nick Hill and Claude Opus 5.5
c055c1e075
[Core] Include the LoRA path in prefix-cache block hashes ( #59335 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 08:37:09 +08:00
Juan Pérez de Algaba and Cursor
cff08b461e
[Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… ( #50300 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-30 08:32:00 -07:00
3d5625cc3c
[Security] Authenticate shared-memory multimodal cache handles ( #59357 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-30 05:16:26 -07:00
713ec07655
[Frontend][RL] Track HTTP weight operation outcomes and concurrency ( #55781 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-30 09:19:43 +00:00
e5f3d08d72
[Frontend][RL] Align sleep-mode API responses and operation metrics ( #52864 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 23:16:38 -07:00
Ting SUN and Flora Feng
329e6eee3f
[Bugfix][Frontend] Report named Anthropic tool calls as tool_use ( #47598 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-30 05:32:38 +00:00
Shijie Lyu
ac68c30872
[Bugfix][Frontend] Ignore reused prompt token ids for media in /v1/responses ( #59173 )
...
Signed-off-by: Shijie Lyu <lshjhf@gmail.com >
2026-09-30 05:21:09 +00:00
42d9f8ccdb
support rl feature : weight checker ( #51350 )
...
Signed-off-by: shiyuan680 <yangcheng104@huawei.com >
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: shiyuan680 <yangcheng104@huawei.com >
Co-authored-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: DeepSeek Harness <noreply@deepseek.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-30 03:13:43 +00:00
Nick Hill and Claude Opus 5.5
105a4e097b
[Bugfix][Frontend] Reject LoRA adapters named after a served model ( #59286 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 18:29:38 -07:00
Flora Feng
7ca31ef599
[Bugfix][Responses API] Build streamed final response from streamed items ( #59307 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-30 00:16:05 +00:00
RongJie G and Flora Feng
0755e69e75
[Bugfix][Responses API] Preserve built-in tool output call IDs ( #55596 )
...
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-29 22:45:14 +00:00
Flora Feng
797505747f
[Bugfix][Frontend] Honor parallel_tool_calls=false in the Responses API ( #59298 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-29 22:36:11 +00:00
