iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
22,441 Commits 3 Branches 0 Tags
d64f6cd08a4eb390664ed5230e6b6ae48b73bfae
Commit Graph
1241 Commits
Author SHA1 Message Date
Yufeng HeandFlora Feng 6f75cc7d5d [Bugfix][Responses API] Reuse streamed item ids in final harmony response (#59859) 
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-04 03:01:04 +00:00
Ting SUNandFlora Feng 50b404e71e [Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams (#47933) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-04 02:57:26 +00:00
aoshen02aoshen02Claude Opus 5.5
 		e319f86f15 [Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-03 14:17:09 +00:00
Martin Hickey bc21cba967 [Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-03 19:38:25 +08:00
Nils Matteson 28556f8600 [Bugfix] Log CRIU failure details before snapshot cleanup (#59661) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 04:02:27 +00:00
Nils Matteson 9fdb174678 [Bugfix] Avoid InfiniBand state in TP1 snapshots (#59699) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
 2026-10-03 03:59:17 +00:00
Wentao Ye f700178f90 [Refactor] Remove dead env and config (#59781) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-10-02 18:07:20 +00:00
Chaoyinglinchaoying6OpenAI Codex
 		592c6f3fbc [Bugfix][Pooling] Handle multimodal cache misses without crashing (#58975) 
Signed-off-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: linchaoying6 <linchaoying6@jd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-10-02 10:14:33 -04:00
AlecBugen ZhaoClaude Opus 5.5
 		9d2a6f52b2 [Rust Frontend] Expose gRPC port in Python vllm serve (#59659) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 08:14:37 +00:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-10-02 03:17:56 +00:00
Yoni GozlanCursorBugen ZhaoClaude Opus 5.5
 		9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 02:33:00 +00:00
Miłosz GrunwaldmicroslawKunshang Ji
 		4caa060053 [TEST][CI] Fix serve rlhf tests subprocess import errors (#59622) 
Signed-off-by: microslaw <you@example.com>
Co-authored-by: microslaw <you@example.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-10-02 09:47:47 +08:00
Anurag SinghAntigravitysfeng33
 		28c57456db [Bugfix][Frontend] Accept Anthropic tool_addition and tool_removal content blocks (#57693) 
Signed-off-by: Anurag Singh <dr.anuragkr@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Antigravity <antigravity@google.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 20:03:36 -04:00
ai-jzai-jzFlora Feng
 		2a52715ef7 [Bugfix][Responses] Use standard reasoning content-part events (#59652) 
Signed-off-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-01 19:20:57 +00:00
Shijie Lyu 9aaef4296f [Bugfix][Frontend] Check reused prompt token ids against the vocab before streaming (#59555) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-10-01 09:26:55 -07:00
Maroon AyoubClaudeRobert ShawRobert Shaw
 		5d9214d6b2 [Bugfix][Frontend] Strip x-anthropic-billing-header billing header from /v1/chatcompletions (#59419) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-01 13:44:40 +00:00
Mark McLoughlinandOpenAI Codex 91fcec1235 [Core][Logging] Add built-in JSON formatter (#58739) 
Add `--logging-config.formatter=json` to enable the python-json-logger
formatter without having to use a complete Python logging dictConfig.

Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-01 09:11:46 +01:00
jiangyunfan1andmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> e12b735109 [Test] Make Anthropic messages test compatible with SDK 1.x via extra_body (#57780) 
Signed-off-by: jiangyunfan1 <jiangyunfan1@h-partners.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 07:38:29 +00:00
aoshen02Codexfh3Claude Fable 5.1
 		4a9d6f9572 [Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 14:43:38 +08:00
Kevin H. LuuandClaude Opus 5.5 8bc31bd210 [CI] Deflake the multi-API-server metrics test and Mooncake PD ports (#59507) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 17:22:58 -07:00
jackLei 21c08ac458 [CI] Deflake shutdown wait-timeout test by synchronizing on request admission (#58125) 
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
 2026-10-01 00:10:03 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Chaojun ZhangandKunshang Ji 02a3c6dfc5 [Test] Skip IPC weight-transfer test on XPU platforms (#59379) 
Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-09-30 21:50:40 +08:00
Clinton ThomasLucas BourtouleCyrus Leung
 		3d5625cc3c [Security] Authenticate shared-memory multimodal cache handles (#59357) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-09-30 05:16:26 -07:00
TaimysCodexLuyi XiaoTaimysHarry MellorClaude Opus 5.5
 		5225f1bacb [MyPy][3/N] Fix MyPy errors in test groups (part 3) (#55939) 
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 11:06:07 +00:00
aoshen02andClaude Opus 5.5 749192ff41 [CI] Skip the IPC weight-checker test on non-CUDA platforms (#59398) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 09:20:32 +00:00
Ronaldaoshen02Claude Opus 5.5aoshen02
 		713ec07655 [Frontend][RL] Track HTTP weight operation outcomes and concurrency (#55781) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 09:19:43 +00:00
Ronaldaoshen02Claude Opus 5.5
 		e5f3d08d72 [Frontend][RL] Align sleep-mode API responses and operation metrics (#52864) 
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:16:38 -07:00
Ting SUNandFlora Feng 329e6eee3f [Bugfix][Frontend] Report named Anthropic tool calls as tool_use (#47598) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-30 05:32:38 +00:00
Shijie Lyu ac68c30872 [Bugfix][Frontend] Ignore reused prompt token ids for media in /v1/responses (#59173) 
Signed-off-by: Shijie Lyu <lshjhf@gmail.com>
 2026-09-30 05:21:09 +00:00
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 834ad456d3 [Refactor] Remove dead tests code (#58916) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-30 04:39:03 +00:00
shiyuan680shiyuan680Ronald1995DeepSeek Harnessaoshen02Claude Opus 5.5aoshen02
 		42d9f8ccdb support rl feature : weight checker (#51350) 
Signed-off-by: shiyuan680 <yangcheng104@huawei.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: shiyuan680 <yangcheng104@huawei.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: DeepSeek Harness <noreply@deepseek.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
 2026-09-30 03:13:43 +00:00
Kevin H. LuuandCodex bec710fccf [CI] Fix mock type narrowing in LoRA serving test (#59344) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 19:55:04 -07:00
Nick HillandClaude Opus 5.5 105a4e097b [Bugfix][Frontend] Reject LoRA adapters named after a served model (#59286) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 18:29:38 -07:00
Flora Feng 7ca31ef599 [Bugfix][Responses API] Build streamed final response from streamed items (#59307) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-30 00:16:05 +00:00
RongJie GandFlora Feng 0755e69e75 [Bugfix][Responses API] Preserve built-in tool output call IDs (#55596) 
Signed-off-by: CorgiBoyG <111257566+CorgiBoyG@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-29 22:45:14 +00:00
Flora Feng 797505747f [Bugfix][Frontend] Honor parallel_tool_calls=false in the Responses API (#59298) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-29 22:36:11 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
Shimi BandielandClaude 4dc57d44c4 [Frontend] Attach resolved logprobs to streaming derender chunks (#55029) 
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 18:43:44 +00:00
aoshen02aoshen02Claude Opus 5.5
 		cd94b21264 [Bugfix][Frontend] Return 400 for malformed RL dev route bodies (#59236) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 16:46:56 +00:00
Shijin ZhangandChauncey d95d1dcfb9 [Bugfix][Frontend] Force reasoning mode for GLM-5.3 chat templates in the GLM MoE parser (#56994) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-29 21:26:34 +08:00
Louie Tsai 208bb38c80 [Bugfix][Rust Frontend] Fix startup with config-only model (#58643) 
Signed-off-by: louie-tsai <louie.tsai@intel.com>
 2026-09-29 10:11:53 +00:00
Shijie LyuShijie LyuClaude
 		c6843d397c [Bugfix][Frontend] Validate reused prompt token ids; render messages for media and echo (#55771) 
Signed-off-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Shijie Lyu <sjlyu@amazon.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-29 10:36:22 +08:00
aoshen02aoshen02Claude Opus 5
 		b721a4c709 [Feature] Add fixed-token prefill scoring (#54335) 
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-28 07:50:09 -07:00
Kevin H. LuuandClaude Opus 5.5 252279314b Revert "[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652)" (#59011) 
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 03:51:17 -07:00
Thang NguyenClaude Opus 5.5khluu
 		413ae4b167 [CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooling) into named jobs (#58652) 
Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com>
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: khluu <khluu000@gmail.com>
 2026-09-28 00:21:08 -07:00
Flora Feng 6f65fde419 [Bugfix][Frontend] Apply Harmony adjust_request in batched chat completions (#58958) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 04:12:04 +00:00
Flora Feng 3184226984 [Bugfix][Frontend] Count Responses reasoning tokens per tool round (#58927) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-28 00:34:38 +00:00
Anton Peganovmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>OpenAI Codex
 		73859fec58 [Frontend] Switch Python Harmony dependency to oss-harmony (#55128) 
Signed-off-by: Anton Peganov <apeganov@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-28 00:43:56 +04:00
Cyrus Leung 379e9a1ea8 [Security] Harden message sanitization (#58832) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-26 16:28:32 +00:00