iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,112 Commits 3 Branches 0 Tags
4e59d53ab534fbb4aaa751eb37ecfbcf438855b5
Commit Graph
1199 Commits
Author SHA1 Message Date
Shijie LyuShijie LyuClaude
 		c6843d397c [Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771) 
Signed-off-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 10:36:22 +08:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Kevin H. LuuandClaude Opus 5.5 252279314b Revert "[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652)" (#59011) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 03:51:17 -07:00
Thang NguyenClaude Opus 5.5khluu
 		413ae4b167 [CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: khluu <khluu000@gmail.com>
 2026-09-28 00:21:08 -07:00
Flora Feng 6f65fde419 [Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 04:12:04 +00:00
Flora Feng 3184226984 [Bugfix][Frontend] Count Responses reasoning tokens per tool round (#58927) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 00:34:38 +00:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
 		73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128) 
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 00:43:56 +04:00
Cyrus Leung 379e9a1ea8 [Security] Harden message sanitization (#58832) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-26 16:28:32 +00:00
Robert ShawandRobert Shaw dfab504333 [Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 14:00:56 +00:00
Robert ShawandRobert Shaw d64f277c54 [Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-09-26 13:50:57 +00:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 09:16:53 -04:00
Juan Pérez de AlgabaandCursor 31f2e70cd3 [Security] Gate per-request multimodal processor kwargs (#58830) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-26 11:10:35 +00:00
Cheng RuiCodexFlora Feng
 		8b47e8b22c [Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067) 
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 14:08:23 -07:00
Denis ErmakovDu BinClaude Opus 5.5Flora Fengmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		7871963fcc [Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call loop (#58551) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Du Bin <8174807+dubin555@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-25 14:00:39 -07:00
Denis ErmakovYanxiao ZhaoPrakhar AgarwalFlora Feng
 		f339e0e757 [Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583) 
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com>
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 16:33:19 +00:00
Taneem IbrahimandWentao Ye bb35e2587b [Pooling] Preserve reranker tokenization with document limits (#57666) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-25 11:04:41 -04:00
Mark McLoughlinandAI Assistant 45d56c0baf [Core] Model console logging as CLI configuration (#57205) 
Add `--logging-config` CLI argument which can be supplied as
JSON or using dotted arguments. The `--log-level` argument
is provided for convenience, and `--log-config-file` is deprecated
in favor of `--logging-config.pylogging_config_file`.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: AI Assistant <noreply@openai.com>
 2026-09-25 11:36:59 +01:00
Robert ShawjrybergRobert ShawClaude Opus 5.5
 		5a2ed2e7ee [Frontend] Handle Disable Thinking in /v1/messages (#58613) 
Signed-off-by: jryberg <johan.ryberg@security.ntt>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: jryberg <johan.ryberg@security.ntt>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-25 12:52:08 +08:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1f0bc49ee6 [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) 
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-25 04:35:10 +00:00
hclClaude Opus 4.8Harry Mellor
 		a7d203efe5 fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) 
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
Signed-off-by: hclsys <chenglunhu@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 08:48:18 -07:00
Wentao Ye 3c42385d7d [Refactor] Remove dead or duplicate tests (#58446) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-24 10:29:56 -04:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
SHREY GAJJAR 5747d4500a [Perf][Frontend] Offload streaming derender detokenization (#57528) 
Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com>
 2026-09-24 10:39:21 +00:00
aoshen02 7f1a5398e9 Fix full logprobs in token-in/token-out responses (#58488) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
 2026-09-24 04:46:16 +00:00
jackQwertyJackChauncey
 		d95a896ea6 [Feature][Frontend] Add DeepSeek-V4 FIM completion rendering (#44229) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 13:02:00 +00:00
Ting SUN b33c8a657d [Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-09-23 14:31:10 +08:00
Juan Pérez de Algaba 4800024b98 [Security] Reject min_tokens that exceeds the filled max_tokens default (#57731) 
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
 2026-09-22 05:50:23 +00:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
Cyrus Leung c8824a1e42 [MM] Move supports_multimodal_inputs and cache out of registry (#57913) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-21 07:43:07 -07:00
Martin Hickey 8b98b7d0b4 [Frontend] Add streaming parity tests and docs for derender (#57922) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-21 14:13:52 +00:00
Hongxin Xuaoshen02OpenAI CodexRoger WangNick HillCodex
 		0b7f11a1ee [AuxOutput] Add block-keyed storage for routed-expert outputs (#45635) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 10:25:05 +00:00
Hongxin Xuaoshen02Codex
 		10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
 2026-09-20 12:21:33 +00:00
Cyrus Leung 56cac80b14 [Deprecation] Remove assistant_token_mask support (#57520) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-19 00:17:51 +08:00
Martin Hickey 70164bdadc [Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-09-18 16:36:27 +02:00
Dakai AnAlanFokCoaoshen02OpenAI Codex
 		092bdd6d57 [Frontend][Core] Add release_kv_cache_memory() API (#44890) 
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-09-18 07:12:12 +08:00
Shimi BandielClaudeCyrus Leungmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		e8a2a0a8bb [Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-17 22:02:00 +00:00
MattandClaude Opus 5 e30bf70c5d [ROCm][CI] Fix Entrypoints Integration (Pooling) tests on TheRock image (#57380) 
Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-17 11:22:39 -05:00
aoshen02 b978df3299 Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302) 2026-09-16 23:31:59 -07:00
aoshen02andClaude Fable 5.1 8538017f4b [Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-17 05:58:56 +00:00
xinnywinneandBenjamin Chislett fbf2c5e8be [Frontend] Add per-request metrics to Responses API (#55084) 
Signed-off-by: wxin <wxin@nvidia.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
 2026-09-16 22:52:05 +00:00
aoshen02 0983aef8da [Frontend] Return prompt metadata from /inference/v1/generate (#53187) 2026-09-16 15:29:07 -07:00
yzong-rhandFlora Feng 5a6ccc5892 [Bugfix] Add Responses cache_write_tokens for API compat and CC parity (#57222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-16 21:52:34 +00:00
Flora Feng e0705731be [Refactor] Remove unused interface methods (#57194) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-16 21:00:48 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
linitra24andJee Jee Li c81eb3e69c [LoRA] Support modules_to_save for sequence classification (#53555) 
Signed-off-by: linitra24 <Joy25810@foxmail.com>
Signed-off-by: linitra24 <renshuang.zhou@daocloud.io>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
 2026-09-16 07:15:42 -07:00
freyfwtCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Jiangyun Zhu
 		18553c5c9f [Kernel] Use Murmur3 RNG for Gumbel sampling (#51367) 
Signed-off-by: freyfwt <110148731+freyfwt@users.noreply.github.com>
Signed-off-by: freyfwt <freytian1996@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-09-15 23:52:26 -07:00
shaohuaxiandCodex e0301afb2a [Bugfix][Responses] Clean up MCP tool sessions once before closing them (#56988) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-16 14:07:06 +08:00
Chauncey f0fbc98c48 [Frontend] Share max_num_queued_reqs across API server processes (#54746) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-09-15 14:50:26 +00:00
ALEX LILUZandJiangyun Zhu 5be911ee87 [Bugfix] Load reasoning parser plugins before headless engine config (#53124) 
Signed-off-by: alexliluz <49665315+alexliluz@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-09-15 18:31:25 +08:00