Sriraj and Claude
43b4aaea3e
[Bugfix][Frontend] Return 4xx for client errors in /cohere/v2/chat ( #60309 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 04:55:34 +00:00
Mohammad Hijjawi and Cyrus Leung
ef014bdf06
[Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses ( #59072 )
...
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-07 04:13:52 +00:00
Sriraj and Flora Feng
db6e3cd8c4
[Bugfix][Frontend] Log the full request body when max_log_len is unset ( #60310 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:42:17 +00:00
Sriraj and Flora Feng
d77cf81156
[Bugfix][Frontend] Honor default penalties in batch chat completions ( #60307 )
...
Signed-off-by: sricursion <sricursion@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-07 00:21:00 +00:00
7b665f7a58
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) ( #58181 )
...
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-06 17:22:47 +00:00
09e0ce3d82
[Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server ( #54835 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-06 18:46:24 +08:00
czhu-cohere and aoshen02
6fddc9b21e
[Profiler] Add per-session profiling controls to Python and Rust frontends ( #57875 )
...
Signed-off-by: czhu <conway.zhu@cohere.com >
Signed-off-by: Conway Zhu <conway.zhu@cohere.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-10-06 05:21:19 +00:00
Mai Hoàng Đức and Flora Feng
5cd8a2626b
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls ( #60036 )
...
Signed-off-by: Mazukiri <maihoangduc04@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 00:06:59 +00:00
Ting SUN and Flora Feng
4a705c7b54
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null ( #47838 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:43:31 +00:00
Yida Weng and Flora Feng
18c8a65edd
[BugFix][Frontend] Pass reasoning_ended through render → generate ( #60059 ) ( #60062 )
...
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 20:39:25 +00:00
Raphaël Rialland and Simon Veitner
c32495d58f
[watermark] compatibility validation ( #56801 )
...
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai >
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Simon Veitner <sveitner@redhat.com >
2026-10-05 14:37:13 -04:00
Tyrone and Harry Mellor
7867d6c52d
[Frontend] Name the served models in the model-not-found 404 ( #59889 )
...
Signed-off-by: TyroneNel <71038642+TyroneNel@users.noreply.github.com >
Signed-off-by: TyroneNel <Tyrone.Nel@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-10-04 20:45:51 +00:00
Stefan Wang and Robert Shaw
b0eb87fe49
[Bugfix][Frontend] Return streaming errors before the first token ( #40986 )
...
Signed-off-by: 1fanwang <1fannnw@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-04 16:42:49 +00:00
Yufeng He and Flora Feng
6f75cc7d5d
[Bugfix][Responses API] Reuse streamed item ids in final harmony response ( #59859 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-04 03:01:04 +00:00
Ting SUN and Flora Feng
50b404e71e
[Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams ( #47933 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-04 02:57:26 +00:00
Martin Hickey
bc21cba967
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) ( #58588 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-03 19:38:25 +08:00
592c6f3fbc
[Bugfix][Pooling] Handle multimodal cache misses without crashing ( #58975 )
...
Signed-off-by: linchaoying6 <linchaoying6@jd.com >
Co-authored-by: linchaoying6 <linchaoying6@jd.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-10-02 10:14:33 -04:00
9d2a6f52b2
[Rust Frontend] Expose gRPC port in Python vllm serve ( #59659 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 08:14:37 +00:00
Martin Hickey
fcfacc3499
[Bugfix][Frontend] Seed derender detokenization from the prompt ( #59046 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-10-02 03:17:56 +00:00
9bd993564f
[Frontend] Parse tool calls and reasoning from checkpoint response templates ( #58604 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 02:33:00 +00:00
28c57456db
[Bugfix][Frontend] Accept Anthropic tool_addition and tool_removal content blocks ( #57693 )
...
Signed-off-by: Anurag Singh <dr.anuragkr@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Antigravity <antigravity@google.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-01 20:03:36 -04:00
2a52715ef7
[Bugfix][Responses] Use standard reasoning content-part events ( #59652 )
...
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-01 19:20:57 +00:00
e9be5323f0
[Perf] Integrate flash-maxsim Triton kernels for late-interaction scoring ( #40337 )
...
Signed-off-by: roi.pony <roi.pony@ibm.com >
Co-authored-by: roi.pony <roi.pony@ibm.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-10-01 14:06:43 -04:00
5d9214d6b2
[Bugfix][Frontend] Strip x-anthropic-billing-header billing header from /v1/chatcompletions ( #59419 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-01 13:44:40 +00:00
Mark McLoughlin and OpenAI Codex
91fcec1235
[Core][Logging] Add built-in JSON formatter ( #58739 )
...
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-10-01 09:11:46 +01:00
4a9d6f9572
[Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs ( #55935 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-01 14:43:38 +08:00
Nick Hill and Claude Opus 5.5
c055c1e075
[Core] Include the LoRA path in prefix-cache block hashes ( #59335 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 08:37:09 +08:00
Juan Pérez de Algaba and Cursor
cff08b461e
[Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… ( #50300 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-30 08:32:00 -07:00
3d5625cc3c
[Security] Authenticate shared-memory multimodal cache handles ( #59357 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-09-30 05:16:26 -07:00
713ec07655
[Frontend][RL] Track HTTP weight operation outcomes and concurrency ( #55781 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-30 09:19:43 +00:00
e5f3d08d72
[Frontend][RL] Align sleep-mode API responses and operation metrics ( #52864 )
...
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 23:16:38 -07:00
Ting SUN and Flora Feng
329e6eee3f
[Bugfix][Frontend] Report named Anthropic tool calls as tool_use ( #47598 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-30 05:32:38 +00:00
Shijie Lyu
ac68c30872
[Bugfix][Frontend] Ignore reused prompt token ids for media in /v1/responses ( #59173 )
...
Signed-off-by: Shijie Lyu <lshjhf@gmail.com >
2026-09-30 05:21:09 +00:00
42d9f8ccdb
support rl feature : weight checker ( #51350 )
...
Signed-off-by: shiyuan680 <yangcheng104@huawei.com >
Signed-off-by: Ronald1995 <ronaldautomobile@163.com >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: shiyuan680 <yangcheng104@huawei.com >
Co-authored-by: Ronald1995 <ronaldautomobile@163.com >
Co-authored-by: DeepSeek Harness <noreply@deepseek.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
2026-09-30 03:13:43 +00:00
Nick Hill and Claude Opus 5.5
105a4e097b
[Bugfix][Frontend] Reject LoRA adapters named after a served model ( #59286 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 18:29:38 -07:00
Flora Feng
7ca31ef599
[Bugfix][Responses API] Build streamed final response from streamed items ( #59307 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-30 00:16:05 +00:00
RongJie G and Flora Feng
0755e69e75
[Bugfix][Responses API] Preserve built-in tool output call IDs ( #55596 )
...
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-29 22:45:14 +00:00
Flora Feng
797505747f
[Bugfix][Frontend] Honor parallel_tool_calls=false in the Responses API ( #59298 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-29 22:36:11 +00:00
Shimi Bandiel and Claude
4dc57d44c4
[Frontend] Attach resolved logprobs to streaming derender chunks ( #55029 )
...
Signed-off-by: Shimi Bandiel <shimib@google.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-29 18:43:44 +00:00
cd94b21264
[Bugfix][Frontend] Return 400 for malformed RL dev route bodies ( #59236 )
...
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 16:46:56 +00:00
af5b4857e1
[Fast Start] Support PP ( #55477 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Xun Sun <UNIDY2002@outlook.com >
2026-09-29 09:37:43 +00:00
Chauncey
61349e342d
[Misc] Avoid repeated warnings for merged Anthropic system messages ( #59172 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-09-29 01:31:22 -07:00
shaohuaxi and Codex
36768d1bfd
[Bugfix][Frontend] Preserve caller parameters in offline pooling ( #58842 )
...
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-29 05:02:59 +00:00
c6843d397c
[Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo ( #55771 )
...
Signed-off-by: Shijie Lyu <sjlyu@amazon.com >
Co-authored-by: Shijie Lyu <sjlyu@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-29 10:36:22 +08:00
b721a4c709
[Feature] Add fixed-token prefill scoring ( #54335 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-28 07:50:09 -07:00
Flora Feng
6f65fde419
[Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions ( #58958 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 04:12:04 +00:00
Flora Feng
027b6f3a22
[Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions ( #58939 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 02:01:35 +00:00
Flora Feng
4d07e90653
[Bugfix][Frontend] Sample batched chat completions from the adjusted requests ( #58929 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 01:46:55 +00:00
Flora Feng
3184226984
[Bugfix][Frontend] Count Responses reasoning tokens per tool round ( #58927 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-28 00:34:38 +00:00
73859fec58
[Frontend] Switch Python Harmony dependency to oss-harmony ( #55128 )
...
Signed-off-by: Anton Peganov <apeganov@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-09-28 00:43:56 +04:00
