iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,316 Commits 3 Branches 0 Tags
bcee730b1a9d25f0fd283a0ef6c19133ebeebf4f
Commit Graph
1924 Commits
Author SHA1 Message Date
Mark McLoughlinandOpenAI Codex 91fcec1235 [Core][Logging] Add built-in JSON formatter (#58739) 
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-01 09:11:46 +01:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Nick HillandClaude Opus 5.5 c055c1e075 [Core] Include the LoRA path in prefix-cache block hashes (#59335) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-01 08:37:09 +08:00
Juan Pérez de AlgabaandCursor cff08b461e [Security] Fix chat template resource-exhaustion DoS (GHSA-4hhp-h66f-… (#50300) 
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-30 08:32:00 -07:00
Clinton ThomasLucas BourtouleCyrus Leung
 		3d5625cc3c [Security] Authenticate shared-memory multimodal cache handles (#59357) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-30 05:16:26 -07:00
Ronaldaoshen02Claude Opus 5.5aoshen02
 		713ec07655 [Frontend][RL] Track HTTP weight operation outcomes and concurrency (#55781) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 09:19:43 +00:00
Ronaldaoshen02Claude Opus 5.5
 		e5f3d08d72 [Frontend][RL] Align sleep-mode API responses and operation metrics (#52864) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:16:38 -07:00
Ting SUNandFlora Feng 329e6eee3f [Bugfix][Frontend] Report named Anthropic tool calls as tool_use (#47598) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-30 05:32:38 +00:00
Shijie Lyu ac68c30872 [Bugfix][Frontend] Ignore reused prompt token ids for media in /v1/responses (#59173) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-09-30 05:21:09 +00:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Nick HillandClaude Opus 5.5 105a4e097b [Bugfix][Frontend] Reject LoRA adapters named after a served model (#59286) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 18:29:38 -07:00
Flora Feng 7ca31ef599 [Bugfix][Responses API] Build streamed final response from streamed items (#59307) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-30 00:16:05 +00:00
RongJie GandFlora Feng 0755e69e75 [Bugfix][Responses API] Preserve built-in tool output call IDs (#55596) 
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-29 22:45:14 +00:00
Flora Feng 797505747f [Bugfix][Frontend] Honor parallel_tool_calls=false in the Responses API (#59298) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-29 22:36:11 +00:00
Shimi BandielandClaude 4dc57d44c4 [Frontend] Attach resolved logprobs to streaming derender chunks (#55029) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 18:43:44 +00:00
aoshen02aoshen02Claude Opus 5.5
 		cd94b21264 [Bugfix][Frontend] Return 400 for malformed RL dev route bodies (#59236) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 16:46:56 +00:00
siyuIsotr0pyXun Sun
 		af5b4857e1 [Fast Start] Support PP (#55477) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Xun Sun <UNIDY2002@outlook.com>
 2026-09-29 09:37:43 +00:00
Chauncey 61349e342d [Misc] Avoid repeated warnings for merged Anthropic system messages (#59172) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-29 01:31:22 -07:00
shaohuaxiandCodex 36768d1bfd [Bugfix][Frontend] Preserve caller parameters in offline pooling (#58842) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 05:02:59 +00:00
Shijie LyuShijie LyuClaude
 		c6843d397c [Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771) 
Signed-off-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 10:36:22 +08:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Flora Feng 6f65fde419 [Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 04:12:04 +00:00
Flora Feng 027b6f3a22 [Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions (#58939) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 02:01:35 +00:00
Flora Feng 4d07e90653 [Bugfix][Frontend] Sample batched chat completions from the adjusted requests (#58929) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 01:46:55 +00:00
Flora Feng 3184226984 [Bugfix][Frontend] Count Responses reasoning tokens per tool round (#58927) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 00:34:38 +00:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
 		73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128) 
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 00:43:56 +04:00
Xun SunandIsotr0py e7900156e1 [Fast Start] Add /health endpoint for the weight cache daemon (#58552) 
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-27 17:48:54 +00:00
Cyrus Leung 379e9a1ea8 [Security] Harden message sanitization (#58832) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-26 16:28:32 +00:00
Robert ShawandRobert Shaw dfab504333 [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 14:00:56 +00:00
Robert ShawandRobert Shaw d64f277c54 [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 13:50:57 +00:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
yzong-rhandBen Browning b6761e8ded [Bugfix] Default missing detail for Responses API input images (#57241) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-25 14:08:55 -07:00
Cheng RuiCodexFlora Feng
 		8b47e8b22c [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) 
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 14:08:23 -07:00
Nick HillandKimi Code de2d478d9a [Bugfix][Frontend] Document 404 response for /generative_scoring (#58788) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-25 14:08:00 -07:00
Denis ErmakovDu BinClaude Opus 5.5Flora Fengmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7871963fcc [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 14:00:39 -07:00
Denis ErmakovYanxiao ZhaoPrakhar AgarwalFlora Feng
 		f339e0e757 [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com>
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 16:33:19 +00:00
Taneem IbrahimandWentao Ye bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 11:04:41 -04:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 50f78084da [Bugfix] Fix generative scoring body cancellation (#57729) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 11:01:31 -04:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
gongwei-130andClaude Fable 5.1 34e104f6fe [gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped (#55102) 
Signed-off-by: Wei Gong <wei@together.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 02:14:32 +00:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 08:48:28 -07:00
hclClaude Opus 4.8Harry Mellor
 		a7d203efe5 fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: hclsys <chenglunhu@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 08:48:18 -07:00
Talor Abramovich 28282ffe34 [Feature][Frontend] Request JSON body debug logging on --enable-log-requests flag (#58163) 
Signed-off-by: talora <talora@nvidia.com>
 2026-09-24 08:08:40 +00:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
jackQwertyJackChauncey
 		d95a896ea6 [Feature][Frontend] Add DeepSeek-V4 FIM completion rendering (#44229) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 13:02:00 +00:00
Ting SUN b33c8a657d [Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-09-23 14:31:10 +08:00
Wentao Ye fae5cbfc8b [Refactor] Remove dead code multiple places (#58002) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-22 16:36:03 -04:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00