iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
21,963 Commits 3 Branches 0 Tags
38cc054fc2269941cd35ecdfd198aacbb2eabfb8
Commit Graph
1184 Commits
Author SHA1 Message Date
Taneem IbrahimandWentao Ye bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 11:04:41 -04:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1f0bc49ee6 [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 04:35:10 +00:00
hclClaude Opus 4.8Harry Mellor
 		a7d203efe5 fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: hclsys <chenglunhu@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 08:48:18 -07:00
Wentao Ye 3c42385d7d [Refactor] Remove dead or duplicate tests (#58446) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-24 10:29:56 -04:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
SHREY GAJJAR 5747d4500a [Perf][Frontend] Offload streaming derender detokenization (#57528) 
Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com>
 2026-09-24 10:39:21 +00:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
jackQwertyJackChauncey
 		d95a896ea6 [Feature][Frontend] Add DeepSeek-V4 FIM completion rendering (#44229) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 13:02:00 +00:00
Ting SUN b33c8a657d [Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-09-23 14:31:10 +08:00
Juan Pérez de Algaba 4800024b98 [Security] Reject min_tokens that exceeds the filled max_tokens default (#57731) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
 2026-09-22 05:50:23 +00:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-21 07:43:07 -07:00
Martin Hickey 8b98b7d0b4 [Frontend] Add streaming parity tests and docs for derender (#57922) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-21 14:13:52 +00:00
Hongxin Xuaoshen02OpenAI CodexRoger WangNick HillCodex
 		0b7f11a1ee [AuxOutput] Add block-keyed storage for routed-expert outputs (#45635) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 10:25:05 +00:00
Hongxin Xuaoshen02Codex
 		10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
 2026-09-20 12:21:33 +00:00
Cyrus Leung 56cac80b14 [Deprecation] Remove assistant_token_mask support (#57520) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-19 00:17:51 +08:00
Martin Hickey 70164bdadc [Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-18 16:36:27 +02:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Shimi BandielClaudeCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e8a2a0a8bb [Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 22:02:00 +00:00
MattandClaude Opus 5 e30bf70c5d [ROCm][CI] Fix Entrypoints Integration (Pooling) tests on TheRock image (#57380) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-17 11:22:39 -05:00
aoshen02 b978df3299 Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302) 2026-09-16 23:31:59 -07:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
xinnywinneandBenjamin Chislett fbf2c5e8be [Frontend] Add per-request metrics to Responses API (#55084) 
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-16 22:52:05 +00:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
yzong-rhandFlora Feng 5a6ccc5892 [Bugfix] Add Responses cache_write_tokens for API compat and CC parity (#57222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-16 21:52:34 +00:00
Flora Feng e0705731be [Refactor] Remove unused interface methods (#57194) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-16 21:00:48 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
linitra24andJee Jee Li c81eb3e69c [LoRA] Support modules_to_save for sequence classification (#53555) 
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-16 07:15:42 -07:00
freyfwtCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jiangyun Zhu
 		18553c5c9f [Kernel] Use Murmur3 RNG for Gumbel sampling (#51367) 
Signed-off-by: freyfwt <110148731+freyfwt@users.noreply.github.com>
Signed-off-by: freyfwt <freytian1996@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-09-15 23:52:26 -07:00
shaohuaxiandCodex e0301afb2a [Bugfix][Responses] Clean up MCP tool sessions once before closing them (#56988) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-16 14:07:06 +08:00
Chauncey f0fbc98c48 [Frontend] Share max_num_queued_reqs across API server processes (#54746) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-15 14:50:26 +00:00
ALEX LILUZandJiangyun Zhu 5be911ee87 [Bugfix] Load reasoning parser plugins before headless engine config (#53124) 
Signed-off-by: alexliluz <49665315+alexliluz@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-09-15 18:31:25 +08:00
Kevin H. LuuOpenAI CodexTyler Michael Smith
 		caf56ce9da [CI] Pass the scale-out endpoint flag in EC E2E (#56819) 
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2026-09-15 12:25:30 +08:00
Martin Hickey 934b1fcbcb [Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-14 10:02:14 +00:00
wang.yuqi 67111973ee [Frontend] Move grpc_server to launchers. (#56746) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-09-14 05:03:08 +00:00
김재억andClaude Opus 5 912dfb3758 [Bugfix][Scoring] Warn when serving original Qwen3 reranker without chat template (#56017) 
Signed-off-by: 김재억 <gadian88@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-11 12:52:45 +00:00
Nicole LiHui 🥜andChauncey 7ef4d9bfed [Bugfix][Responses] Fix browser.find action type (#55305) 
Signed-off-by: nicole-lihui <nicole.li@daocloud.io>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-11 12:07:04 +00:00
shaohuaxiandCodex a2bc2ffb2c [Bugfix][Pooling] Restore token limits for offline Jina scoring (#56415) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-11 10:26:44 +00:00
wang.yuqi fe4d4109c2 [Frontend][last/N] Move all non-OpenAI content out of the OpenAI folder. (#56369) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-09-11 05:33:26 +00:00
Colin McNamara 360f33f61c [Bugfix] Set stop_sequence explicitly in streaming message_delta event (#55325) 
Signed-off-by: Colin McNamara <colin@2cups.com>
 2026-09-10 18:21:21 -04:00
shaohuaxiandCodex ae71862c51 [Bugfix][Frontend] Check EC requirements for each metadata item (#56070) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-10 19:23:34 +00:00
Jiatai WangandOpenAI Codex a36dfc93cb [Bugfix][Multimodal] Restore cached audio inputs with UUIDs (#56310) 
Signed-off-by: JiataiWang <wangjiatai@proton.me>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-10 19:20:03 +00:00
Tianyu Guo 479611c0c6 [Frontend][EPD] Use JSON arrays for multimodal metadata (#56090) 
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
 2026-09-10 05:04:14 +00:00
Raphaël RiallandOpenAI Codexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		ea40bb9e90 [feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-09 21:00:15 -07:00
Nicolò Lucchesizhouyou9505Cursor Grok 4.6OpenAI Codex
 		9ffb8cea96 [CI] Add e2e test for scale-out EC connector flow (#54973) 
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: zhouyou9505 <zhouyou9505@gmail.com>
Co-authored-by: Cursor Grok 4.6 <noreply@cursor.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-09 07:38:32 -07:00
Adababyandshaolila 5133e1d285 [Frontend] Migrate input-validation errors to VLLMValidationError (harmony/mistral/chat_utils/params) (#55735) 
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
Co-authored-by: shaolila <shaolila@buaa.edu.cn>
 2026-09-08 06:45:40 +00:00
Taneem Ibrahim fdfa0a1659 [Pooling] Honor request_id from request bodies (#55665) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-07 19:55:20 -04:00