iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,580 Commits 3 Branches 0 Tags
ef014bdf06d4fd468e2bcef63aed98bddaeda123
Commit Graph
1256 Commits
Author SHA1 Message Date
Mohammad HijjawiandCyrus Leung ef014bdf06 [Bugfix][Frontend] Return logprobs in Cohere Chat v2 responses (#59072) 
Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-07 04:13:52 +00:00
SrirajandFlora Feng db6e3cd8c4 [Bugfix][Frontend] Log the full request body when max_log_len is unset (#60310) 
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-07 00:42:17 +00:00
SrirajandFlora Feng d77cf81156 [Bugfix][Frontend] Honor default penalties in batch chat completions (#60307) 
Signed-off-by: sricursion <sricursion@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-07 00:21:00 +00:00
Yin LiYin LiClaude
 		7b665f7a58 [Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181) 
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 17:22:47 +00:00
Shimi BandielClaudeCyrus Leung
 		09e0ce3d82 [Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server (#54835) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-06 18:46:24 +08:00
roikoren755andzjy0516 e11962fc1c [Feat][Mamba2] Enable internal prefill checkpoints (#57329) 
Signed-off-by: Roi Koren <roik@nvidia.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-10-06 07:53:51 +00:00
sungbin1015andClaude 1e5d0ea888 [CI/Build] Fix incorrect CLI arg validation tests in test_cli_args.py (#46779) 
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-10-06 05:58:47 +00:00
czhu-cohereandaoshen02 6fddc9b21e [Profiler] Add per-session profiling controls to Python and Rust frontends (#57875) 
Signed-off-by: czhu <conway.zhu@cohere.com>
Signed-off-by: Conway Zhu <conway.zhu@cohere.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-10-06 05:21:19 +00:00
Mai Hoàng ĐứcandFlora Feng 5cd8a2626b [Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls (#60036) 
Signed-off-by: Mazukiri <maihoangduc04@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 00:06:59 +00:00
Ting SUNandFlora Feng 4a705c7b54 [Bugfix][Frontend] Preserve logprobs when top_logprobs is null (#47838) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 22:43:31 +00:00
Yida WengandFlora Feng 18c8a65edd [BugFix][Frontend] Pass reasoning_ended through render → generate (#60059) (#60062) 
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 20:39:25 +00:00
Raphaël RiallandandSimon Veitner c32495d58f [watermark] compatibility validation (#56801) 
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
 2026-10-05 14:37:13 -04:00
Jee Jee Li f9c9e8ac24 [LoRA] Remove tensorizer (#60024) 
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-10-05 07:01:56 -07:00
Stefan WangandRobert Shaw b0eb87fe49 [Bugfix][Frontend] Return streaming errors before the first token (#40986) 
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-04 16:42:49 +00:00
Aarushi JainandAndreas Karatzas bdd31c31de [CI] Stop requiring both API servers to record weight sync metrics (#59521) 
Signed-off-by: aarushjain29 <aarushi.jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-04 14:14:27 +00:00
Yufeng HeandFlora Feng 6f75cc7d5d [Bugfix][Responses API] Reuse streamed item ids in final harmony response (#59859) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-04 03:01:04 +00:00
Ting SUNandFlora Feng 50b404e71e [Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams (#47933) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-04 02:57:26 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Martin Hickey bc21cba967 [Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-03 19:38:25 +08:00
Nils Matteson 28556f8600 [Bugfix] Log CRIU failure details before snapshot cleanup (#59661) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 04:02:27 +00:00
Nils Matteson 9fdb174678 [Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 03:59:17 +00:00
Wentao Ye f700178f90 [Refactor] Remove dead env and config (#59781) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-02 18:07:20 +00:00
Chaoyinglinchaoying6OpenAI Codex
 		592c6f3fbc [Bugfix][Pooling] Handle multimodal cache misses without crashing (#58975) 
Signed-off-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-02 10:14:33 -04:00
AlecBugen ZhaoClaude Opus 5.5
 		9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 08:14:37 +00:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-02 03:17:56 +00:00
Yoni GozlanCursorBugen ZhaoClaude Opus 5.5
 		9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 02:33:00 +00:00
Miłosz GrunwaldmicroslawKunshang Ji
 		4caa060053 [TEST][CI] Fix serve rlhf tests subprocess import errors (#59622) 
Signed-off-by: microslaw <you@example.com>
Co-authored-by: microslaw <you@example.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-02 09:47:47 +08:00
Anurag SinghAntigravitysfeng33
 		28c57456db [Bugfix][Frontend] Accept Anthropic tool_addition and tool_removal content blocks (#57693) 
Signed-off-by: Anurag Singh <dr.anuragkr@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Antigravity <antigravity@google.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 20:03:36 -04:00
ai-jzai-jzFlora Feng
 		2a52715ef7 [Bugfix][Responses] Use standard reasoning content-part events (#59652) 
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-01 19:20:57 +00:00
Shijie Lyu 9aaef4296f [Bugfix][Frontend] Check reused prompt token ids against the vocab before streaming (#59555) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-10-01 09:26:55 -07:00
Maroon AyoubClaudeRobert ShawRobert Shaw
 		5d9214d6b2 [Bugfix][Frontend] Strip x-anthropic-billing-header billing header from /v1/chatcompletions (#59419) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-01 13:44:40 +00:00
Mark McLoughlinandOpenAI Codex 91fcec1235 [Core][Logging] Add built-in JSON formatter (#58739) 
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-01 09:11:46 +01:00
jiangyunfan1andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e12b735109 [Test] Make Anthropic messages test compatible with SDK 1.x via extra_body (#57780) 
Signed-off-by: jiangyunfan1 <jiangyunfan1@h-partners.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 07:38:29 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Kevin H. LuuandClaude Opus 5.5 8bc31bd210 [CI] Deflake the multi-API-server metrics test and Mooncake PD ports (#59507) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 17:22:58 -07:00
jackLei 21c08ac458 [CI] Deflake shutdown wait-timeout test by synchronizing on request admission (#58125) 
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
 2026-10-01 00:10:03 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Chaojun ZhangandKunshang Ji 02a3c6dfc5 [Test] Skip IPC weight-transfer test on XPU platforms (#59379) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-30 21:50:40 +08:00
Clinton ThomasLucas BourtouleCyrus Leung
 		3d5625cc3c [Security] Authenticate shared-memory multimodal cache handles (#59357) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-30 05:16:26 -07:00
TaimysCodexLuyi XiaoTaimysHarry MellorClaude Opus 5.5
 		5225f1bacb [MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939) 
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 11:06:07 +00:00
aoshen02andClaude Opus 5.5 749192ff41 [CI] Skip the IPC weight-checker test on non-CUDA platforms (#59398) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 09:20:32 +00:00
Ronaldaoshen02Claude Opus 5.5aoshen02
 		713ec07655 [Frontend][RL] Track HTTP weight operation outcomes and concurrency (#55781) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 09:19:43 +00:00
Ronaldaoshen02Claude Opus 5.5
 		e5f3d08d72 [Frontend][RL] Align sleep-mode API responses and operation metrics (#52864) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:16:38 -07:00
Ting SUNandFlora Feng 329e6eee3f [Bugfix][Frontend] Report named Anthropic tool calls as tool_use (#47598) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-30 05:32:38 +00:00
Shijie Lyu ac68c30872 [Bugfix][Frontend] Ignore reused prompt token ids for media in /v1/responses (#59173) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-09-30 05:21:09 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 834ad456d3 [Refactor] Remove dead tests code (#58916) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 04:39:03 +00:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Kevin H. LuuandCodex bec710fccf [CI] Fix mock type narrowing in LoRA serving test (#59344) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 19:55:04 -07:00
Nick HillandClaude Opus 5.5 105a4e097b [Bugfix][Frontend] Reject LoRA adapters named after a served model (#59286) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 18:29:38 -07:00
Flora Feng 7ca31ef599 [Bugfix][Responses API] Build streamed final response from streamed items (#59307) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-30 00:16:05 +00:00