 Maxwill Lin
|
e48f2aa4ca
|
[Bugfix][Frontend] Emit a content block for empty Anthropic completions (#46525)
Signed-off-by: EazyReal <8047065+EazyReal@users.noreply.github.com>
|
2026-06-24 00:04:26 +00:00 |
|
|
Guy StoneandClaude
|
3cc871aaf1
|
[Perf] Skip detokenization in online beam search (#46422)
Signed-off-by: Guy Stone <guys@spotify.com>
Signed-off-by: Guy Stone <guystone3@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-06-23 15:46:09 -07:00 |
|
|
Martin HickeyandHarry Mellor
|
275b43183c
|
[MyPy] Fix mypy for vllm/benchmarks (#39896)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-23 15:22:29 +00:00 |
|
|
yzong-rhandBen Browning
|
f59db63732
|
[Bugfix] GPT-OSS Autodrop reasoning in Response API and cleanup (#45048)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-23 09:36:33 -04:00 |
|
|
Muhammad FawazandMahad Rehmann
|
1bf149f334
|
Filter Pydantic-internal markers from validation error param (#46457)
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com>
Co-authored-by: Mahad Rehmann <114791389+mahadrehmann@users.noreply.github.com>
|
2026-06-23 13:20:50 +00:00 |
|
|
yzong-rhandBen Browning
|
2a675a7b9f
|
[Bugfix] Responses API assistant EasyInputMessageParam input (#44361)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-23 08:54:46 -04:00 |
|
|
wang.yuqi
|
31ca9504b1
|
[Frontend] Split ServingRender into renderer and entrypoint. (#44285)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-23 11:19:09 +00:00 |
|
|
|
6c427dd401
|
[BugFix] Omit empty tool_calls from OpenAI chat responses (#44105)
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-06-23 13:43:53 +08:00 |
|
|
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
3ce5823762
|
[Refactor] Responses API parser state into conversation context (#46030)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-23 13:42:58 +08:00 |
|
|
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f2069b005b
|
[Pooling] Validate non-negative rerank top_n (#46119)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-22 11:40:47 -04:00 |
|
|
wang.yuqi
|
1eb2cc961e
|
[Frontend] Refactor ServingTokenization entrypoint. (#46022)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-22 06:27:58 +00:00 |
|
|
|
3e6e33526d
|
[Disagg] return routed_experts on streaming generate responses (#44638)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-06-21 07:37:10 -07:00 |
|
|
aoshen02andMartin Hickey
|
f57ac274b2
|
[Render] Add reasoning/tool parsing to /derender + fix byte-fallback FFFD (#45919)
Signed-off-by: aoshen524 <aoshen524@gmail.com>
Co-authored-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-06-20 19:43:32 -04:00 |
|
|
shuoming zhangandTyler Michael Smith
|
891cc4b9c5
|
[Frontend] Report cache usage in Anthropic /v1/messages API (#40912)
Signed-off-by: mistral0105 <zhangshuoming17@mails.ucas.ac.cn>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-06-20 21:12:48 +00:00 |
|
|
|
ebfbcfe46a
|
Stop setting CUDA_VISIBLE_DEVICES internally in vLLM, add device_ids arg (#45026)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: kourosh hakhamaneshi <kouroshHakha@users.noreply.github.com>
|
2026-06-20 13:38:10 -07:00 |
|
|
Sumanth R Hegde
|
7ff7f5c8eb
|
Revert "Fix Stale Encoder Cache After Weight Update" (#46125)
|
2026-06-20 07:09:09 -07:00 |
|
|
Muhammad FawazandMahad Durrani
|
69bdd34542
|
[Bugfix] Fall back to Pydantic loc for param in validation errors (#46038)
Signed-off-by: professorsab <135441198+professorsab@users.noreply.github.com>
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com>
|
2026-06-19 19:11:11 +08:00 |
|
|
|
4ce2d01453
|
fix(anthropic): auto-detect template support for mid-conversation system messages (#46025)
Signed-off-by: felix0080 <felix0080@users.noreply.github.com>
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: felix0080 <felix0080@users.noreply.github.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-18 16:19:11 -04:00 |
|
|
littlecircle0730
|
08985351f3
|
Fix Stale Encoder Cache After Weight Update (#45093)
Signed-off-by: littlecircle0730 <littlecircle0730@gmail.com>
|
2026-06-18 09:32:10 +00:00 |
|
|
MrFanandfelix0080
|
1e9f04da14
|
fix(anthropic): preserve inline system message position for prefix caching (#44602)
Signed-off-by: felix0080 <felix0080@users.noreply.github.com>
Co-authored-by: felix0080 <felix0080@users.noreply.github.com>
|
2026-06-18 15:58:11 +08:00 |
|
|
Ting SUNandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
702214146c
|
[Bugfix][Frontend] Fix Anthropic count_tokens decorator order driving server load negative (#44725)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-17 23:56:46 -07:00 |
|
|
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
e2c58570ea
|
[Rust Frontend] Support hybrid/external DP LB in Python supervised bootstrap (#45805)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-17 07:32:40 +00:00 |
|
|
Taneem Ibrahim
|
43fa24e832
|
[Misc] Validate Cohere Embed Mixed Content Payloads (#45873)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-06-17 06:57:12 +00:00 |
|
|
Taneem Ibrahim
|
81d8f4ebac
|
[Misc] Added validation for Cohere /v2/embed input field exclusivity (#45640)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-06-16 05:42:43 +00:00 |
|
|
Zang Peiyu
|
25ee659db0
|
Fix parallel_tool_calls: null treated as false instead of default true (#44955)
Signed-off-by: factnn <166481866+factnn@users.noreply.github.com>
|
2026-06-15 21:14:10 +00:00 |
|
|
Flora Feng
|
cd9078fe59
|
[Frontend] Skip structural tags for auto tool_choice without strict mode (#45600)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-15 19:55:31 +00:00 |
|
|
yzong-rhandBen Browning
|
51ec5cf08f
|
[Bugfix] Chat Completions Harmony Refactor Clean up (#45464)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-15 14:45:19 -04:00 |
|
|
Flora Feng
|
0d80979644
|
[Chore] Consolidate reasoning/tool parser attributes into unified Parser in chat serving (#45548)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-15 11:16:45 -04:00 |
|
|
Juan Pérez de Algaba
|
b997071ec4
|
(security) Enforce audio upload size limit before full file materialization (#45510)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-06-15 10:25:24 +00:00 |
|
|
Ting SUN
|
48df95c43e
|
[Feature][Frontend] Report multimodal token counts in usage.prompt_tokens_details (#45458)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-15 05:20:58 +00:00 |
|
|
Roger WangandClaude Opus 4.8
|
e8d3e22c88
|
Fix included router missing path for FastAPI >=0.137 (#45629)
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-06-15 04:28:52 +00:00 |
|
|
Taneem Ibrahim
|
2c764c089a
|
Added real /v1/embeddings support for messages + chat_template_kw (#45173)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-06-15 09:08:10 +08:00 |
|
|
Andreas Karatzas
|
9261dbbc55
|
Treat null completion max_tokens like the default (#45491)
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
|
2026-06-13 09:34:09 +00:00 |
|
|
Martin HickeyandIsotr0py
|
43f0e024bc
|
[Render] Add /derender endpoints for disaggregated postprocessing (#43606)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-06-13 13:55:33 +08:00 |
|
|
WEI CHENG CHIU
|
17ee5b1ac5
|
[Bugfix] Set type/role explicitly in streaming message_start event (#45376)
Signed-off-by: Wayne Chiu <waynehacking8@gmail.com>
|
2026-06-13 01:40:50 +00:00 |
|
|
Flora Feng
|
6e4a547176
|
[Refactor] Deprecate ResponsesParser wrapper, inline parsing into ParsableContext (#45431)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-12 16:15:41 -04:00 |
|
|
Guan-Ming (Wesley) Chiu
|
c7aa3d2630
|
[Core] Support structured outputs for beam search (#35022)
Signed-off-by: Guan-Ming (Wesley) Chiu <guanmingchiu@gmail.com>
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
|
2026-06-12 06:56:25 -07:00 |
|
|
Juan Pérez de Algaba
|
f715f25f29
|
Fix misleading error for audio duration limit rejection (#45113)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-06-12 09:58:08 +00:00 |
|
|
|
2043258dec
|
[Frontend] Support strict mode for tool calling (#45003)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-12 07:51:48 +00:00 |
|
|
Ekagra Ranjanandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
e0b9fb1290
|
[ASR] Optimize CPU preproc to get 2.5x RTFx via multi-threading (#44612)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-11 21:05:11 -07:00 |
|
|
|
42ae5e7ac6
|
[Bugfix] Fix --enable-prompt-tokens-details omitting zero cached tokens (#44383)
Signed-off-by: Sasindharan Sankar <sasindharansankar@email.com>
Co-authored-by: Sasindharan Sankar <sasindharansankar@email.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-06-11 20:37:42 -07:00 |
|
|
yzong-rh
|
e0871ad225
|
[Refactor] Chat Completions Streaming Harmony Refactor and Bugfixes (#45104)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-06-12 01:09:47 +00:00 |
|
|
yzong-rh
|
f712fd0d7d
|
[Refactor] Chat Completions Harmony Refactor, non-streaming path. (#45171)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-06-11 21:18:30 +00:00 |
|
|
Ben Browning
|
235b63c004
|
[Bugfix] Fix Anthropic tool_use content handling dropping args (#45287)
Signed-off-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-11 20:01:29 +00:00 |
|
|
Harry Mellor
|
03878d1c22
|
Deprecations for v0.23 and v0.24 (#44992)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-11 14:35:38 +00:00 |
|
|
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
9492362972
|
[Security] Apply sanitize_message to Anthropic and STT error paths (#45119)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-11 10:05:34 +00:00 |
|
|
Flora Feng
|
3a04061701
|
[Refactor][Parser] Unify Response API to use parser.parse() like Chat Completion API (#45190)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-11 04:37:51 +00:00 |
|
|
ankrovvandFlora Feng
|
248e33c40d
|
[Bugfix][Responses API] Set id on function_call item in streaming done event (#44608)
Signed-off-by: Aniruddh Krovvidi <aniruddh.krovvidi@oracle.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-06-11 03:52:42 +00:00 |
|
|
yzong-rhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
6ec7dcd641
|
[Frontend][Metrics] Add vllm:tool_call_parser_invocations_total Prometheus metric (#44448)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-10 10:29:11 -04:00 |
|
|
Varun Vinayak ShenoyandCodex
|
5828a205ef
|
Fix Harmony tool descriptions for optional fields (#44686)
Signed-off-by: Varun Shenoy <varun.vinayak.shenoy@oracle.com>
Co-authored-by: Codex <codex@openai.com>
|
2026-06-09 23:29:22 -07:00 |
|