lzhan011
fa99a6fea6
[Bugfix][Security] Bound the validation-error response body ( #54684 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
2026-09-01 06:51:16 +00:00
Amy (Yu-Hsuan) Lin
923949e6e3
[Feat] Add request-level preemption count histogram metric ( #49984 )
2026-08-31 23:04:54 -04:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
stefankoncarevic
5f213ed159
[ROCm][CI] Warm up the RLHF dev server before the pause/resume timing checks ( #53594 )
...
Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com >
2026-08-28 09:07:04 +00:00
Moe Huzaifa
3e833933dc
[Bugfix][Frontend] Keep credentials out of the Rust frontend launch log ( #53738 )
...
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com >
2026-08-25 16:32:12 +00:00
Moe Huzaifa
d3e2888c75
[Bugfix][Frontend] Redact hf_token in the non-default args log ( #53625 )
...
Signed-off-by: mhuzaifa3 <mhuzaifa3@outlook.com >
2026-08-25 03:06:46 +00:00
wang.yuqi
9cef631f30
[Frontend] Move run_batch.py out openai folder ( #53500 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-24 15:04:10 +00:00
wang.yuqi
eac636a7fa
[Frontend] Move api_server.py out openai folder ( #52131 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-19 09:43:46 +00:00
LuckyStep
9a9aa2b017
[Bugfix] Redact api_key in non-default args log ( #52523 )
...
Signed-off-by: LuckyStep <67696304+Andy365-365@users.noreply.github.com >
2026-08-19 15:52:53 +08:00
Rajath Pai
b01728b088
[Bugfix] Return 4xx for client-caused errors in /detokenize ( #52622 )
...
Signed-off-by: rajathpi <rajathpai2000@gmail.com >
2026-08-18 07:07:15 +00:00
flb_
d785eb51ce
[Test] Add pause/resume E2E tests ( #52144 )
...
Signed-off-by: flb_ <floatlibai@gmail.com >
2026-08-18 14:46:44 +08:00
c89d692cb5
[Build] Propagate vLLM version to Rust binaries ( #52593 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-17 23:37:37 -07:00
wang.yuqi
93550cc4cd
[Frontend] Consolidate entrypoint middleware ( #52309 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-17 06:17:15 +00:00
Do_it_now_!
70aaec832b
[Bugfix][Anthropic] Return 4xx for client-caused errors in /v1/messages ( #52246 )
...
Signed-off-by: Do_it_now_! <lizhengcheng233@gmail.com >
2026-08-16 01:05:17 -07:00
wang.yuqi
b8165e5e58
[Frontend] Consolidate entrypoint exception handler ( #52261 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-08-14 09:27:24 +00:00
yang rui
152c913a63
[Frontend] Log output token IDs at DEBUG level ( #52098 )
...
Signed-off-by: ruirui6946 <142162413+ruirui6946@users.noreply.github.com >
2026-08-13 13:11:51 +00:00
Zach Zhu
58f9659397
[Frontend][Core] Standardize request error handling with VLLMError hierarchy ( #49665 )
...
Signed-off-by: Zach Zhu <zzqshu@126.com >
2026-07-29 04:20:40 +00:00
Harry Mellor and Andreas Karatzas
ca0defa343
Make bare hugging_face imports forbidden ( #49726 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-25 04:18:39 -07:00
wang.yuqi
61c9ef986a
[Frontend] Parallelize preprocessing within the same request for pooling models online serving. ( #49153 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-07-22 10:56:23 +00:00
Muhammad Fawaz and Mahad Durrani
e87521626f
Sanitize server file paths from validation error responses ( #46415 )
...
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com >
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com >
2026-07-09 17:46:29 +08:00
Zach Zhu
5d5fab0061
[Bugfix][Frontend] Fix http_requests_total metric recording some 4xx errors as 5xx ( #44303 )
...
Signed-off-by: Zach Zhu <zzqshu@126.com >
2026-07-08 05:33:21 +00:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
e55cc59e52
[Rust Frontend][CI] Unblock more end-to-end test cases ( #47735 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-07 08:27:20 +00:00
wang.yuqi
a14f57a3ac
[Frontend] Refine the entrypoint class's inheritance hierarchy. ( #47498 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-07-03 10:50:06 +00:00
Jyothirmai Kottu and Claude
dee5da1dec
[Test] Run SageMaker handler-override tests in-process via TestClient ( #47250 )
...
Signed-off-by: Jyothirmai Kottu <jkottu@amazon.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-01 09:14:00 +00:00
wang.yuqi
3483240b7e
[Frontend] Consolidate scale out entrypoints ( #44512 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-29 03:18:53 -07:00
Ranran and Benjamin Chislett
09841ae705
[Render][Speculator] Add return_loss_mask to render endpoint for training data generation ( #46846 )
...
Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-06-28 00:07:33 -07:00
Hyunkyun Moon
950ee4c2e4
[API] Add token offsets to render endpoints (/v1/.../render) ( #44226 )
...
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com >
2026-06-26 05:02:52 -07:00
Muhammad Fawaz and Mahad Rehmann
1bf149f334
Filter Pydantic-internal markers from validation error param ( #46457 )
...
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com >
Co-authored-by: Mahad Rehmann <114791389+mahadrehmann@users.noreply.github.com >
2026-06-23 13:20:50 +00:00
wang.yuqi
31ca9504b1
[Frontend] Split ServingRender into renderer and entrypoint. ( #44285 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-23 11:19:09 +00:00
wang.yuqi
1eb2cc961e
[Frontend] Refactor ServingTokenization entrypoint. ( #46022 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-22 06:27:58 +00:00
aoshen02 and Martin Hickey
f57ac274b2
[Render] Add reasoning/tool parsing to /derender + fix byte-fallback FFFD ( #45919 )
...
Signed-off-by: aoshen524 <aoshen524@gmail.com >
Co-authored-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-06-20 19:43:32 -04:00
Muhammad Fawaz and Mahad Durrani
69bdd34542
[Bugfix] Fall back to Pydantic loc for param in validation errors ( #46038 )
...
Signed-off-by: professorsab <135441198+professorsab@users.noreply.github.com >
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com >
2026-06-19 19:11:11 +08:00
Martin Hickey and Isotr0py
43f0e024bc
[Render] Add /derender endpoints for disaggregated postprocessing ( #43606 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-06-13 13:55:33 +08:00
Andreas Karatzas
1033ffac2e
[CI] Wait for SSL cert refresher events in the test ( #45489 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-06-13 04:57:18 +00:00
Ethan Feng
b7f9b6ab27
[Metrics] Add group-aware KV cache capacity to vllm:cache_config_info ( #42206 )
...
The startup log already reports the correct group-aware KV cache capacity for
hybrid models, but Prometheus did not expose matching info in 'vllm:cache_config_info`.
This PR adds kv_cache_size_tokens and kv_cache_max_concurrency.
Signed-off-by: Ethan Feng <ethan.fengch@gmail.com >
2026-06-12 11:49:44 +00:00
42ae5e7ac6
[Bugfix] Fix --enable-prompt-tokens-details omitting zero cached tokens ( #44383 )
...
Signed-off-by: Sasindharan Sankar <sasindharansankar@email.com >
Co-authored-by: Sasindharan Sankar <sasindharansankar@email.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-06-11 20:37:42 -07:00
Juan Pérez de Algaba and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
9492362972
[Security] Apply sanitize_message to Anthropic and STT error paths ( #45119 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-11 10:05:34 +00:00
wang.yuqi
d01d0b4646
[Frontend] Consolidate online serving utils. ( #44479 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-04 06:49:31 +00:00
Ted Mostly
0c1e6f63f5
[Bugfix] Fix VLLMNotFoundError when using LoRA adapter name in poolin… ( #44410 )
...
Signed-off-by: Ted Mostly <wanghenshui@qq.com >
2026-06-04 02:22:03 +00:00
wang.yuqi
b623f7ea95
[Frontend] Consolidate dev entrypoints. ( #44170 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-02 06:30:21 -07:00
Andreas Karatzas
5940590855
[ROCm][CI] Stabilize 400 error return code for invalid schema inputs ( #43016 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-05-24 10:06:49 +00:00
Kevin H. Luu and Claude Opus 4.6
ebbfb34e3e
[Test] Replace zephyr-7b-beta (7B) with SmolLM2-135M in tokenization test ( #43085 )
...
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-05-21 01:57:47 -07:00
Aaron Hao and Isotr0py
2d6b3489b9
[R3] Add routed experts to openai entrypoint ( #38939 )
...
Signed-off-by: ahao-anyscale <ahao@anyscale.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2026-05-20 09:07:59 -07:00
Sage
a78b842d0e
[Bugfix] Fix top logprobs token placeholders in /inference/v1/generate ( #42887 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-05-19 10:21:49 +00:00
39c67d714e
fix: add API key authorization to /v2 endpoints ( #42594 )
...
Signed-off-by: DustHunter <dusthunter@126.com >
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-05-16 01:29:27 +00:00
hallerite and Claude Opus 4.7
97c4317bf5
[Bugfix][Frontend] Default max_tokens server-side on /inference/v1/generate ( #42329 )
...
Signed-off-by: hallerite <git@hallerite.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-13 11:16:46 +02:00
Chenguang Zheng and OpenAI Codex
07351e0883
[Feature] Warm up readonly multimodal processor during renderer startup ( #40797 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-04-25 03:57:41 +00:00
Nithin Chalapathi and Cyrus Leung
80b18230e0
[Frontend] Add multimodal support to /inference/v1/generate endpoint ( #38405 )
...
Signed-off-by: Nithin Chalapathi <nithin.ch10@gmail.com >
Signed-off-by: Nithin Chalapathi <nithinc@berkeley.edu >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-04-17 20:31:56 -07:00
5c18b961d6
[Core][Metrics] expose waiting request breakdown via labeled metric (capacity/deferred) ( #38435 )
...
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com >
Signed-off-by: Mark McLoughlin <markmc@redhat.com >
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com >
Co-authored-by: Mark McLoughlin <markmc@redhat.com >
2026-04-13 15:30:55 -04:00
wang.yuqi
66c079ae83
[Frontend][4/n] Improve pooling entrypoints | pooling. ( #39153 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-09 10:09:45 +00:00
lzhan011