iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,112 Commits 3 Branches 0 Tags
4e59d53ab534fbb4aaa751eb37ecfbcf438855b5
Commit Graph
1905 Commits
Author SHA1 Message Date
Shijie LyuShijie LyuClaude
 		c6843d397c [Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771) 
Signed-off-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 10:36:22 +08:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Flora Feng 6f65fde419 [Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 04:12:04 +00:00
Flora Feng 027b6f3a22 [Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions (#58939) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 02:01:35 +00:00
Flora Feng 4d07e90653 [Bugfix][Frontend] Sample batched chat completions from the adjusted requests (#58929) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 01:46:55 +00:00
Flora Feng 3184226984 [Bugfix][Frontend] Count Responses reasoning tokens per tool round (#58927) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 00:34:38 +00:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
 		73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128) 
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 00:43:56 +04:00
Xun SunandIsotr0py e7900156e1 [Fast Start] Add /health endpoint for the weight cache daemon (#58552) 
Signed-off-by: Xun Sun <UNIDY2002@outlook.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-27 17:48:54 +00:00
Cyrus Leung 379e9a1ea8 [Security] Harden message sanitization (#58832) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-26 16:28:32 +00:00
Robert ShawandRobert Shaw dfab504333 [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 14:00:56 +00:00
Robert ShawandRobert Shaw d64f277c54 [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 13:50:57 +00:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
yzong-rhandBen Browning b6761e8ded [Bugfix] Default missing detail for Responses API input images (#57241) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-25 14:08:55 -07:00
Cheng RuiCodexFlora Feng
 		8b47e8b22c [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) 
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 14:08:23 -07:00
Nick HillandKimi Code de2d478d9a [Bugfix][Frontend] Document 404 response for /generative_scoring (#58788) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
 2026-09-25 14:08:00 -07:00
Denis ErmakovDu BinClaude Opus 5.5Flora Fengmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7871963fcc [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 14:00:39 -07:00
Denis ErmakovYanxiao ZhaoPrakhar AgarwalFlora Feng
 		f339e0e757 [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com>
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 16:33:19 +00:00
Taneem IbrahimandWentao Ye bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 11:04:41 -04:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 50f78084da [Bugfix] Fix generative scoring body cancellation (#57729) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 11:01:31 -04:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Isotr0py 24928a4d62 [UX][Frontend] Introduce vllm preload cli for fast restart (#56680) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-25 06:56:39 +00:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
gongwei-130andClaude Fable 5.1 34e104f6fe [gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped (#55102) 
Signed-off-by: Wei Gong <wei@together.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-25 02:14:32 +00:00
Harry MellorandClaude Opus 5.5 d636456b1f [Frontend] Remove the slow tokenizer mode (#58545) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 08:48:28 -07:00
hclClaude Opus 4.8Harry Mellor
 		a7d203efe5 fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: hclsys <chenglunhu@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 08:48:18 -07:00
Talor Abramovich 28282ffe34 [Feature][Frontend] Request JSON body debug logging on --enable-log-requests flag (#58163) 
Signed-off-by: talora <talora@nvidia.com>
 2026-09-24 08:08:40 +00:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
jackQwertyJackChauncey
 		d95a896ea6 [Feature][Frontend] Add DeepSeek-V4 FIM completion rendering (#44229) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 13:02:00 +00:00
Ting SUN b33c8a657d [Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-09-23 14:31:10 +08:00
Wentao Ye fae5cbfc8b [Refactor] Remove dead code multiple places (#58002) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-22 16:36:03 -04:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
sfigs 21aa17c128 [Bugfix][Frontend] Accept diarized transcription responses in run-batch (#57948) 
Signed-off-by: Sergio Figueras <sergio@yourecm.com>
 2026-09-21 14:55:38 +00:00
Taneem Ibrahim 887cf91e30 [Pooling] Fix normalization of chunked long-text embeddings (#57498) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-21 10:46:39 -04:00
Martin Hickey 8b98b7d0b4 [Frontend] Add streaming parity tests and docs for derender (#57922) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-21 14:13:52 +00:00
Hongxin Xuaoshen02Codex
 		10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
 2026-09-20 12:21:33 +00:00
hangangqiangmukeshbhandarkaraoshen02Claude Fable 5.1aoshen02
 		90531daf6d [Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-18 18:11:05 -07:00
Cyrus Leung 56cac80b14 [Deprecation] Remove assistant_token_mask support (#57520) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-19 00:17:51 +08:00
Martin Hickey 70164bdadc [Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-18 16:36:27 +02:00
lazymioClaude Opus 5Bugen Zhao
 		dee37d8911 [Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 04:32:39 +00:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Shimi BandielClaudeCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e8a2a0a8bb [Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 22:02:00 +00:00
aoshen02 b978df3299 Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302) 2026-09-16 23:31:59 -07:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
xinnywinneandBenjamin Chislett fbf2c5e8be [Frontend] Add per-request metrics to Responses API (#55084) 
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-16 22:52:05 +00:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
yzong-rhandFlora Feng 5a6ccc5892 [Bugfix] Add Responses cache_write_tokens for API compat and CC parity (#57222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-16 21:52:34 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Misha Goin 0384e72693 [UX] Add thinking support to vllm chat (#57045) 2026-09-16 09:54:10 +01:00
wenjinhust 0d8173d153 [Bugfix][Frontend] Lazy-import model_hosting_container_standards to prevent log suppression (#51366) 
Signed-off-by: wenjinhust <wenjin.sh@huawei.com>
 2026-09-16 08:25:49 +00:00