|
f2376cdc4d
|
[Fast Start] Preload the FlashInfer autotune table on the weight cache daemon (#60085)
Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-10 15:48:06 +00:00 |
|
|
Kay YanandCodex
|
c41b2639e2
|
[Bugfix][Responses API] Complete cache-write token accounting (#60973)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-10 09:08:36 +00:00 |
|
|
wang.yuqi
|
c1692c85f1
|
[Misc] mv chat_utils.py to renderers. (#60522)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-10 06:50:34 +00:00 |
|
|
Chauncey
|
fa5cfd48d5
|
[Refactor] Deduplicate trace header handling in BaseServing (#60828)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-10-10 05:27:58 +00:00 |
|
|
Misha GoinandClaude Opus 5.5
|
29e4d2d42e
|
[Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 21:22:35 -07:00 |
|
|
|
3c802af3cf
|
[Feat][Frontend] structured decisions: DiffusionGemma read strategy (#60466)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-10 09:33:22 +08:00 |
|
|
premsurawutandClaude Opus 5.5
|
c811943137
|
[Bugfix][Frontend] Reject beam search with echo and logprobs in /v1/completions (#60876)
Signed-off-by: surawut.j <surawut.jirasaktavee@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 18:06:26 +00:00 |
|
|
|
1a3a7a8f01
|
[Bugfix] Treat max_tokens as output upper bound, not input-space reservation (#57035)
Signed-off-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com>
Signed-off-by: YuvrajSinghBhadoria2 <yuvrajsinghbhado2030@gmail.com>
Signed-off-by: Yuvraj Singh Bhadoria <yuvrajsinghbhado2030@gmail.com>
Co-authored-by: YuvrajSinghBhadoria2 <245782664+YuvrajSinghBhadoria2@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-10-10 01:15:22 +08:00 |
|
|
Taewoong KimandCyrus Leung
|
75e3b17c05
|
[Frontend] Add cache salt support to generative scoring (#60841)
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-09 16:07:50 +00:00 |
|
|
Damien LaineandCyrus Leung
|
2186b89a9a
|
[Bugfix][Model] Voxtral realtime: fix boot OOM and engine crash at max_model_len (#45022)
Signed-off-by: Damien Laine <damien.laine@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-09 17:40:29 +08:00 |
|
|
Taewoong KimandChauncey
|
18c19eaf23
|
[Frontend] Add cache salt and tracing to structured decisions (#60786)
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-09 08:53:50 +00:00 |
|
|
Maroon AyoubandClaude
|
8a03ccf7b3
|
[Frontend] Add return_mm_kwargs to render requests (#60728)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-09 08:14:13 +00:00 |
|
|
|
8cbd5d0300
|
[Frontend] structured decisions: noul and score question types (#60696)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: DeepSeek <noreply@deepseek.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-09 04:48:16 +00:00 |
|
|
|
fa312090cb
|
[Frontend] Accept each connection on the least loaded API worker (#60723)
Signed-off-by: hao-aaron <aaronhao@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 04:40:13 +00:00 |
|
|
Vineeth Sai VarikuntlaandFlora Feng
|
112a282915
|
[Bugfix][Frontend] Keep streamed completion text_offset in step with tokens (#60668)
Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 21:20:43 -07:00 |
|
|
sungbin1015andFlora Feng
|
2ddde19e8e
|
[Frontend] Support min_p in the Responses API (#48084)
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 20:38:17 +00:00 |
|
|
|
ae9adec905
|
[Frontend] Expose /v1/systemone without an opt-in flag (#60677)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-08 20:21:05 +00:00 |
|
|
dingdangmaoandFlora Feng
|
bab0ac61f8
|
[Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions (#59263)
Signed-off-by: apex-mochen <2756823972@qq.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 19:32:46 +00:00 |
|
|
Yeonwoo SungandFlora Feng
|
56e6da2c47
|
[Bugfix][Frontend] Honor stop_token_ids on the Responses API (#56592)
Signed-off-by: YeonwooSung <neos960518@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 18:47:30 +00:00 |
|
|
Francisco Javier ArceoandMisha Goin
|
705e124c05
|
[Bugfix] Followup to #59299: preserve Unicode in structured decision state prompts (#60651)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-08 11:00:03 -07:00 |
|
|
|
d33872dfeb
|
[Feat] Structured decisions endpoint (/v1/systemone) (#59299)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-08 14:48:12 +00:00 |
|
|
wang.yuqi
|
459764104d
|
[Frontend] Move offline_utils.py to entrypoints/common. (#58052)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-08 09:56:05 +00:00 |
|
|
Harry MellorandClaude Opus 5.5
|
21d9758459
|
Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 02:30:52 -07:00 |
|
|
Ting SUN
|
0f112d1800
|
[Bugfix][Pooling] Preserve cache_salt for chunked embeddings (#47696)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-10-08 07:33:28 +00:00 |
|
|
|
f226af0e11
|
[Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners (#54113)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-08 06:25:27 +00:00 |
|
|
|
4187ade7b8
|
[Frontend] Consolidate rl entrypoints. (#57849)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 21:37:11 -07:00 |
|
|
bnkaiteamandFlora Feng
|
3722645539
|
[Bugfix][Frontend] Accept successful 2xx responses for batch output uploads (#60507)
Signed-off-by: bnkaiteam <bnkaiteam@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 02:32:35 +00:00 |
|
|
|
2dd887426e
|
[Bugfix][Frontend] Do not modify caller params in LLM.score() (#59959)
Signed-off-by: 叮当猫 <2756823972@qq.com>
Signed-off-by: apex-mochen <2756823972@qq.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-10-08 09:03:08 +08:00 |
|
|
Kang NahoonandFlora Feng
|
1995c0fcd0
|
[Bugfix][Frontend] Apply output_config and thinking in Anthropic count_tokens (#59180)
Signed-off-by: Kang Nahoon <ejrtks102020@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 00:24:20 +00:00 |
|
|
YueshenZandsfeng33
|
ed825aafc0
|
[Bugfix][Frontend] Fix Responses namespace aliases in named tool calling (#59723)
Signed-off-by: Shen <312058264+YueshenZ@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
|
2026-10-08 00:07:52 +00:00 |
|
|
SrirajandClaude
|
43b4aaea3e
|
[Bugfix][Frontend] Return 4xx for client errors in /cohere/v2/chat (#60309)
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-07 04:55:34 +00:00 |
|
|
Mohammad HijjawiandCyrus Leung
|
ef014bdf06
|
[Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses (#59072)
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-07 04:13:52 +00:00 |
|
|
SrirajandFlora Feng
|
db6e3cd8c4
|
[Bugfix][Frontend] Log the full request body when max_log_len is unset (#60310)
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-07 00:42:17 +00:00 |
|
|
SrirajandFlora Feng
|
d77cf81156
|
[Bugfix][Frontend] Honor default penalties in batch chat completions (#60307)
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-07 00:21:00 +00:00 |
|
|
|
7b665f7a58
|
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-06 17:22:47 +00:00 |
|
|
|
09e0ce3d82
|
[Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server (#54835)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-06 18:46:24 +08:00 |
|
|
czhu-cohereandaoshen02
|
6fddc9b21e
|
[Profiler] Add per-session profiling controls to Python and Rust frontends (#57875)
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-06 05:21:19 +00:00 |
|
|
Mai Hoàng ĐứcandFlora Feng
|
5cd8a2626b
|
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls (#60036)
Signed-off-by: Mazukiri <maihoangduc04@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-06 00:06:59 +00:00 |
|
|
Ting SUNandFlora Feng
|
4a705c7b54
|
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null (#47838)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-05 22:43:31 +00:00 |
|
|
Yida WengandFlora Feng
|
18c8a65edd
|
[BugFix][Frontend] Pass reasoning_ended through render → generate (#60059) (#60062)
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-05 20:39:25 +00:00 |
|
|
Raphaël RiallandandSimon Veitner
|
c32495d58f
|
[watermark] compatibility validation (#56801)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
|
2026-10-05 14:37:13 -04:00 |
|
|
TyroneandHarry Mellor
|
7867d6c52d
|
[Frontend] Name the served models in the model-not-found 404 (#59889)
Signed-off-by: TyroneNel <71038642+TyroneNel@users.noreply.github.com>
Signed-off-by: TyroneNel <Tyrone.Nel@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-10-04 20:45:51 +00:00 |
|
|
Stefan WangandRobert Shaw
|
b0eb87fe49
|
[Bugfix][Frontend] Return streaming errors before the first token (#40986)
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-04 16:42:49 +00:00 |
|
|
Yufeng HeandFlora Feng
|
6f75cc7d5d
|
[Bugfix][Responses API] Reuse streamed item ids in final harmony response (#59859)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-04 03:01:04 +00:00 |
|
|
Ting SUNandFlora Feng
|
50b404e71e
|
[Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams (#47933)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-04 02:57:26 +00:00 |
|
|
Martin Hickey
|
bc21cba967
|
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-03 19:38:25 +08:00 |
|
|
|
592c6f3fbc
|
[Bugfix][Pooling] Handle multimodal cache misses without crashing (#58975)
Signed-off-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-10-02 10:14:33 -04:00 |
|
|
|
9d2a6f52b2
|
[Rust Frontend] Expose gRPC port in Python vllm serve (#59659)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-02 08:14:37 +00:00 |
|
|
Martin Hickey
|
fcfacc3499
|
[Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-02 03:17:56 +00:00 |
|
|
|
9bd993564f
|
[Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-02 02:33:00 +00:00 |
|