iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,979 Commits 3 Branches 0 Tags
2d7dd0e5c1f9800a3b8c8f38fd832ee335fa7fba
Commit Graph
1889 Commits
Author SHA1 Message Date
Denis ErmakovYanxiao ZhaoPrakhar AgarwalFlora Feng
 		f339e0e757 [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com>
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 16:33:19 +00:00
Taneem IbrahimandWentao Ye bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 11:04:41 -04:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 50f78084da [Bugfix] Fix generative scoring body cancellation (#57729) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 11:01:31 -04:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
gongwei-130andClaude Fable 5.1 34e104f6fe [gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped (#55102) 
Signed-off-by: Wei Gong <wei@together.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 02:14:32 +00:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 08:48:28 -07:00
hclClaude Opus 4.8Harry Mellor
 		a7d203efe5 fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: hclsys <chenglunhu@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 08:48:18 -07:00
Talor Abramovich 28282ffe34 [Feature][Frontend] Request JSON body debug logging on --enable-log-requests flag (#58163) 
Signed-off-by: talora <talora@nvidia.com>
 2026-09-24 08:08:40 +00:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
jackQwertyJackChauncey
 		d95a896ea6 [Feature][Frontend] Add DeepSeek-V4 FIM completion rendering (#44229) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 13:02:00 +00:00
Ting SUN b33c8a657d [Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-09-23 14:31:10 +08:00
Wentao Ye fae5cbfc8b [Refactor] Remove dead code multiple places (#58002) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-22 16:36:03 -04:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
sfigs 21aa17c128 [Bugfix][Frontend] Accept diarized transcription responses in run-batch (#57948) 
Signed-off-by: Sergio Figueras <sergio@yourecm.com>
 2026-09-21 14:55:38 +00:00
Taneem Ibrahim 887cf91e30 [Pooling] Fix normalization of chunked long-text embeddings (#57498) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-21 10:46:39 -04:00
Martin Hickey 8b98b7d0b4 [Frontend] Add streaming parity tests and docs for derender (#57922) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-21 14:13:52 +00:00
Hongxin Xuaoshen02Codex
 		10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
 2026-09-20 12:21:33 +00:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
Cyrus Leung 56cac80b14 [Deprecation] Remove assistant_token_mask support (#57520) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-19 00:17:51 +08:00
Martin Hickey 70164bdadc [Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-18 16:36:27 +02:00
lazymioClaude Opus 5Bugen Zhao
 		dee37d8911 [Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 04:32:39 +00:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Shimi BandielClaudeCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e8a2a0a8bb [Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 22:02:00 +00:00
aoshen02 b978df3299 Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302) 2026-09-16 23:31:59 -07:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
xinnywinneandBenjamin Chislett fbf2c5e8be [Frontend] Add per-request metrics to Responses API (#55084) 
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-16 22:52:05 +00:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
yzong-rhandFlora Feng 5a6ccc5892 [Bugfix] Add Responses cache_write_tokens for API compat and CC parity (#57222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-16 21:52:34 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Misha Goin 0384e72693 [UX] Add thinking support to vllm chat (#57045) 2026-09-16 09:54:10 +01:00
wenjinhust 0d8173d153 [Bugfix][Frontend] Lazy-import model_hosting_container_standards to prevent log suppression (#51366) 
Signed-off-by: wenjinhust <wenjin.sh@huawei.com>
 2026-09-16 08:25:49 +00:00
shaohuaxiandCodex e0301afb2a [Bugfix][Responses] Clean up MCP tool sessions once before closing them (#56988) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-16 14:07:06 +08:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 1fd119def5 [Pooling] Cap max-length padding for chunked embeddings (#56505) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-15 20:31:51 -04:00
Taneem Ibrahim 2fcc524788 [Bugfix] Avoid nested score cancellation handlers for /v1/score alias (#57024) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-15 15:41:54 -04:00
ALEX LILUZandJiangyun Zhu 5be911ee87 [Bugfix] Load reasoning parser plugins before headless engine config (#53124) 
Signed-off-by: alexliluz <49665315+alexliluz@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-09-15 18:31:25 +08:00
shaohuaxiandCodex ca67438c08 [Bugfix][Frontend] Handle aborted requests in beam search (#56249) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-15 08:17:30 +00:00
Miguel a72c046463 [Bugfix] Avoid nested rerank cancellation handlers (#55602) 
Signed-off-by: Miguel Garcia <miguelgarciaroman8@gmail.com>
 2026-09-15 03:19:08 +00:00
Martin Hickey 934b1fcbcb [Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-14 10:02:14 +00:00
Hung Hoang (黄兴） 663d7f679e [Bugfix] Fix --lora-modules name=path parsing when path contains '=' (#53353) 
Signed-off-by: hungh <hungh@nvidia.com>
 2026-09-13 23:55:21 -07:00
Alexander LeeandOpenAI Codex 10e8d5614c [Bugfix][Beam Search] Respect skip_special_tokens during decoding (#56211) 
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-14 05:49:44 +00:00
wang.yuqi 67111973ee [Frontend] Move grpc_server to launchers. (#56746) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-09-14 05:03:08 +00:00
Taneem Ibrahim 2e9f7bb454 [Pooling] Report actual input token usage for scoring APIs (#56573) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-13 09:20:21 -04:00
Walter Beller-MoralesandFlora Feng 756794a9a7 [Frontend] create unified Cohere parser (#56392) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-12 04:04:46 -04:00
김재억andClaude Opus 5 912dfb3758 [Bugfix][Scoring] Warn when serving original Qwen3 reranker without chat template (#56017) 
Signed-off-by: 김재억 <gadian88@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 12:52:45 +00:00
Nicole LiHui 🥜andChauncey 7ef4d9bfed [Bugfix][Responses] Fix browser.find action type (#55305) 
Signed-off-by: nicole-lihui <nicole.li@daocloud.io>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-11 12:07:04 +00:00
shaohuaxiandCodex a2bc2ffb2c [Bugfix][Pooling] Restore token limits for offline Jina scoring (#56415) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-11 10:26:44 +00:00
wang.yuqi fe4d4109c2 [Frontend][last/N] Move all non-OpenAI content out of the OpenAI folder. (#56369) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-09-11 05:33:26 +00:00