iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
20,492 Commits 3 Branches 0 Tags
3cc015acd265d0e4e4c9d0a73bf7e5d5a5e66eef
Commit Graph
193 Commits
Author SHA1 Message Date
Connor CarpenterandBugen Zhao 3cc015acd2 [Rust Frontend][gRPC] Enforce LoRA path validation across transports (#53756) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-27 16:00:12 -07:00
Bugen ZhaoandNick Hill 9db222c6be [Rust Frontend] Align OpenAI request and response edge cases (#53218) 
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-27 19:23:38 +00:00
Jung JiyuTyler Michael SmithOpenAI Codex
 		3fd72ffd7b [Bugfix][Rust Frontend] Fix LogprobsTensors wire schema mismatch (#53939) 
Signed-off-by: jungjiyu <libraryofjiyu@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-26 16:16:43 -07:00
Jiahao LiangandChauncey 59d7fc92ea [Bugfix][DeepSeek V4] Handle trailing system messages in prompt rendering (#51262) 
Signed-off-by: jiahaoliang <gzliangjiahao@gmail.com>
Signed-off-by: Jiahao Liang <gzliangjiahao@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-08-25 07:49:44 +00:00
Connor CarpenterBugen Zhaomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		1f9444a34f [Rust Frontend][gRPC] Add LoRA lifecycle control (#52840) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-25 04:13:19 +00:00
Biswa Panda 9dba2c9b37 [Rust Frontend][RL]: report engine world size over gRPC (#53204) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
 2026-08-23 02:31:23 +00:00
Taneem Ibrahim a014e35f38 [Pooling] Fix Rust pooling endpoint for benchmark (#53352) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-08-22 15:45:09 -04:00
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 704f12aa1e [Pooling] Report input throughput for batched requests (#53213) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-22 09:01:40 -04:00
Bugen ZhaoandOpenAI Codex 08b73d07cc [Frontend] Prevent Kimi K3 reserved markers in response text (#52889) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-21 09:06:52 -07:00
Roy Wang df344bf36f [Rust][Benchmark] Load HF datasets from parquet shards via hf-hub, fixing truncated-cache sampling (#51570) 2026-08-21 05:07:43 +00:00
ReidandBugen Zhao 2adf4b9e2a [Rust Frontend] Fix Kimi K3 reasoning_effort="none" handling (#53043) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-08-21 02:47:07 +00:00
Bugen Zhao df3b3422b4 [Rust Frontend] Add HY3 unified parser and local XGrammar structural-tag builder (#53054) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 19:13:54 -07:00
Bugen ZhaoOpenAI CodexNick Hill
 		83c5d59209 [Rust Frontend] Replace external protoc with pure Rust lib protox (#52892) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-20 18:43:51 -07:00
Bugen Zhao f32b17b6d6 [Rust Frontend] Support --generation-config vllm (#53044) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 17:31:25 -07:00
Matthew KotilaReed MeyersonClaude Opus 4.8
 		7cfb97e337 [Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915) 
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-20 15:42:49 -07:00
Sagemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>Bugen Zhao
 		727274a75b [Rust Frontend] Fix Qwen parser auto-detection (#51169) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
 2026-08-20 09:39:05 +00:00
Roy WangClaude Fable 5Bugen Zhao
 		963fcfa48c [Rust][Benchmark] Align speed-bench CLI flags with Python and add flag parity test (#51592) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 08:50:51 +00:00
Quentin GallouédecandBugen Zhao 16cfe728d8 [Bugfix][Rust Frontend] Reject n > 1 in the /inference/v1/generate route (#52844) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-20 07:41:06 +00:00
Tyler Michael SmithandOpenAI Codex aabc1a0a0b [Bugfix][Benchmark] Check readiness before tokenizer init in rust vllm-bench (#51863) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-19 02:52:25 +00:00
Biswa Pandaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6a391a931a [Rust Frontend][RL] add routed expert prompt offset (#52703) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 20:04:28 +00:00
Connor CarpenterandBugen Zhao d75136c030 [Rust Frontend] Wait for all utility calls to finish (#52671) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-18 11:23:34 -07:00
Bugen ZhaoandNick Hill 41f179b57a [Rust Frontend] Simplify data-parallel size ownership (#52575) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-18 09:06:05 +00:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
Woosuk KwonBugen ZhaoOpenAI Codex
 		e0e5a7fb28 [Rust Frontend] Fix GLM-5.2 chat template rendering parity (#51426) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
 2026-08-18 03:16:54 +00:00
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f08a95f8d8 [Rust Frontend][gRPC] Advertise LoRA capabilities (#52031) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-17 13:27:56 -07:00
Biswa Panda 42156466db [Rust Frontend][gRPC] Preserve skip_special_tokens decoding option (#52384) 
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
 2026-08-15 08:54:58 +00:00
Connor CarpenterandNick Hill f473870ecf [Rust Frontend][gRPC] Add RL lifecycle control (#51316) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-08-14 11:32:39 -07:00
vx120aoshen02CodexClaude Opus 4.6OpenAI Codex
 		50ba4bc6b2 [Feature] Mask Replay (#49577) 
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-13 08:44:31 +00:00
WillZZZyCyrus LeungClaude
 		3962042304 [Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift (#46747) (#46747) 
Signed-off-by: William Zhao <williamzh@meta.com>
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-12 12:06:57 +08:00
Clinton ThomasLucas Bourtoulemergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		b64a2708b0 Bound generation inputs before expensive work (#51447) 
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 22:03:50 +08:00
aoshen02Claude Opus 4.6mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		33584901ba [Frontend] Add content_parts to /inference/v1/generate for raw multim… (#51478) 
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 10:00:12 +08:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> f1e921b6d6 [Rust Frontend] Support dynamic tools from developer messages (#51144) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-10 18:30:38 -07:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> d8c70f2243 [Rust Frontend] Upgrade MiniJinja to 2.22 & remove method lookup workaround (#51235) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-11 00:23:24 +00:00
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> fa722b9f01 [Rust Frontend][gRPC] Add explicit data-parallel rank routing (#51178) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-10 15:05:24 -07:00
Sage 865781e627 [Rust Frontend] Add standalone Rust renderer (#50289) 2026-08-06 11:23:26 +00:00
Chauncey 22170354d8 [Feature] Parse request priority from HTTP header (#51089) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-05 21:10:29 -07:00
SageandBugen Zhao 5bf32605a9 [Rust Frontend] Deduplicate request preprocessing for /tokenize (#50448) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-05 08:27:35 +00:00
Bugen Zhao 7743486190 [Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings (#50580) 2026-08-04 08:51:32 -07:00
Connor CarpenterandBugen Zhao e98a8774cb [Rust Frontend][gRPC] Add multimodal image inference (#50368) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-04 09:34:31 +00:00
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 0b1c151cbc [Rust Frontend] Align tool rendering for Kimi K3 (#50540) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-08-04 08:03:46 +00:00
Reid 72cd5424da [Rust][Benchmark] Preserve UTF-8 across benchmark stream chunks (#50868) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-08-04 06:15:02 +00:00
Karen Chung f57123aa2d feat(frontend): session id plumbing into requests (#48048) 
Signed-off-by: Karen Chung <karenc@nvidia.com>
 2026-08-03 22:18:58 +00:00
zcxGGmuandzq c4e9f09de7 [Bugfix][Frontend] Reject empty gRPC stop strings (#50746) 
Signed-off-by: zq <zhouquan1511@163.com>
Co-authored-by: zq <zhouquan1511@163.com>
 2026-08-03 20:50:15 +00:00
Reid 62195e9784 [Rust][Benchmark] Prevent invalid token IDs in random benchmarks (#50058) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
 2026-08-01 03:20:29 +00:00
Bugen Zhao ab98034d4c [Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness (#50420) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-31 10:02:17 +08:00
Samuel LaferriereandClaude d6938b7408 [Bugfix][Rust Frontend] Select earliest-completing stop string (#50200) 
Signed-off-by: Samuel Laferriere <9342524+samlaf@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-31 00:33:19 +00:00
Bugen ZhaoandOpenAI Codex e9096fcb12 [Rust Frontend] Improve startup failure and readiness logs (#50406) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-30 14:18:17 -07:00
Bugen Zhao 629a938a92 [Frontend] Preserve bare Inkling text in Python and Rust parsers (#50403) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-30 12:25:10 -07:00
Connor CarpenterandNick Hill 48fc2e2797 feat(grpc): add KV event source discovery (#50033) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
 2026-07-29 13:46:40 -07:00
ReidandBugen Zhao 242c591d5a [Rust Frontend] Send multimodal tensors in auxiliary frames (#49341) 
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-29 12:21:09 +00:00