727274a75b
[Rust Frontend] Fix Qwen parser auto-detection ( #51169 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-08-20 09:39:05 +00:00
963fcfa48c
[Rust][Benchmark] Align speed-bench CLI flags with Python and add flag parity test ( #51592 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: esmeetu <jasonailu87@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-20 08:50:51 +00:00
Quentin Gallouédec and Bugen Zhao
16cfe728d8
[Bugfix][Rust Frontend] Reject n > 1 in the /inference/v1/generate route ( #52844 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-20 07:41:06 +00:00
Tyler Michael Smith and OpenAI Codex
aabc1a0a0b
[Bugfix][Benchmark] Check readiness before tokenizer init in rust vllm-bench ( #51863 )
...
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-19 02:52:25 +00:00
Biswa Panda and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
6a391a931a
[Rust Frontend][RL] add routed expert prompt offset ( #52703 )
...
Signed-off-by: Biswa Panda <biswa.panda@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-18 20:04:28 +00:00
Connor Carpenter and Bugen Zhao
d75136c030
[Rust Frontend] Wait for all utility calls to finish ( #52671 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-08-18 11:23:34 -07:00
Bugen Zhao and Nick Hill
41f179b57a
[Rust Frontend] Simplify data-parallel size ownership ( #52575 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-18 09:06:05 +00:00
c89d692cb5
[Build] Propagate vLLM version to Rust binaries ( #52593 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-17 23:37:37 -07:00
e0e5a7fb28
[Rust Frontend] Fix GLM-5.2 chat template rendering parity ( #51426 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-08-18 03:16:54 +00:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f08a95f8d8
[Rust Frontend][gRPC] Advertise LoRA capabilities ( #52031 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-17 13:27:56 -07:00
Biswa Panda
42156466db
[Rust Frontend][gRPC] Preserve skip_special_tokens decoding option ( #52384 )
...
Signed-off-by: Biswa Panda <biswa.panda@gmail.com >
2026-08-15 08:54:58 +00:00
Connor Carpenter and Nick Hill
f473870ecf
[Rust Frontend][gRPC] Add RL lifecycle control ( #51316 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-14 11:32:39 -07:00
50ba4bc6b2
[Feature] Mask Replay ( #49577 )
...
Signed-off-by: vx120 <893600387@qq.com >
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-13 08:44:31 +00:00
3962042304
[Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift ( #46747 ) ( #46747 )
...
Signed-off-by: William Zhao <williamzh@meta.com >
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-12 12:06:57 +08:00
b64a2708b0
Bound generation inputs before expensive work ( #51447 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 22:03:50 +08:00
33584901ba
[Frontend] Add content_parts to /inference/v1/generate for raw multim… ( #51478 )
...
Signed-off-by: Allen Shen <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 10:00:12 +08:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f1e921b6d6
[Rust Frontend] Support dynamic tools from developer messages ( #51144 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-10 18:30:38 -07:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
d8c70f2243
[Rust Frontend] Upgrade MiniJinja to 2.22 & remove method lookup workaround ( #51235 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 00:23:24 +00:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fa722b9f01
[Rust Frontend][gRPC] Add explicit data-parallel rank routing ( #51178 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-10 15:05:24 -07:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
635dd6aae6
[Build][gRPC] Publish protobuf schemas to Buf ( #51276 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-10 13:34:39 -07:00
Sage
865781e627
[Rust Frontend] Add standalone Rust renderer ( #50289 )
2026-08-06 11:23:26 +00:00
Chauncey
22170354d8
[Feature] Parse request priority from HTTP header ( #51089 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-05 21:10:29 -07:00
Sage and Bugen Zhao
5bf32605a9
[Rust Frontend] Deduplicate request preprocessing for /tokenize ( #50448 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-05 08:27:35 +00:00
Bugen Zhao
7743486190
[Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings ( #50580 )
2026-08-04 08:51:32 -07:00
Connor Carpenter and Bugen Zhao
e98a8774cb
[Rust Frontend][gRPC] Add multimodal image inference ( #50368 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-04 09:34:31 +00:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
0b1c151cbc
[Rust Frontend] Align tool rendering for Kimi K3 ( #50540 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-04 08:03:46 +00:00
Reid
72cd5424da
[Rust][Benchmark] Preserve UTF-8 across benchmark stream chunks ( #50868 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-08-04 06:15:02 +00:00
Karen Chung
f57123aa2d
feat(frontend): session id plumbing into requests ( #48048 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-08-03 22:18:58 +00:00
zcxGGmu and zq
c4e9f09de7
[Bugfix][Frontend] Reject empty gRPC stop strings ( #50746 )
...
Signed-off-by: zq <zhouquan1511@163.com >
Co-authored-by: zq <zhouquan1511@163.com >
2026-08-03 20:50:15 +00:00
Reid
62195e9784
[Rust][Benchmark] Prevent invalid token IDs in random benchmarks ( #50058 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-08-01 03:20:29 +00:00
Bugen Zhao
ab98034d4c
[Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness ( #50420 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-31 10:02:17 +08:00
Samuel Laferriere and Claude
d6938b7408
[Bugfix][Rust Frontend] Select earliest-completing stop string ( #50200 )
...
Signed-off-by: Samuel Laferriere <9342524+samlaf@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-31 00:33:19 +00:00
Bugen Zhao and OpenAI Codex
e9096fcb12
[Rust Frontend] Improve startup failure and readiness logs ( #50406 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-30 14:18:17 -07:00
Bugen Zhao
629a938a92
[Frontend] Preserve bare Inkling text in Python and Rust parsers ( #50403 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-30 12:25:10 -07:00
Connor Carpenter and Nick Hill
48fc2e2797
feat(grpc): add KV event source discovery ( #50033 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-29 13:46:40 -07:00
Reid and Bugen Zhao
242c591d5a
[Rust Frontend] Send multimodal tensors in auxiliary frames ( #49341 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-29 12:21:09 +00:00
Maria Guevara
c44e191b01
[Rust Frontend] Add --limit-mm-per-prompt support ( #49604 )
...
Signed-off-by: Maria Guevara <kawaiiplush14@gmail.com >
2026-07-29 17:21:41 +08:00
Bugen Zhao
2899dca843
[Model] Add Kimi K3 support: Rust frontend [1/2] ( #50104 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-28 14:31:13 -07:00
Bugen Zhao
d552a68645
[Rust Frontend] Extract shared tracing setup logic into vllm-tracing ( #50129 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-28 19:31:15 +00:00
Reid
912d6b619d
[Rust Frontend] Align sampling validation with Python ( #47494 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-28 12:02:00 +00:00
Reid
bf9f23003c
[Rust Frontend] Fix finish reason for named tool choices ( #49496 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-28 10:53:18 +00:00
Bugen Zhao and OpenAI Codex
5ed3faa43d
[Rust Frontend] Add ordinary-text tokenizer encoding ( #49992 )
...
Co-authored-by: OpenAI Codex <noreply@openai.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-28 15:46:51 +08:00
Connor Carpenter and Nick Hill
e3c2fc3b3c
[Rust Frontend][gRPC] Add server and model discovery ( #49491 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-27 09:53:27 -07:00
Nick Hill and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
59a6b0411d
[Core] Fix internal LB load-balancing ( #49204 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 14:00:46 +00:00
Bugen Zhao and OpenAI Codex
fd9d2ede6f
[Rust Frontend] Keep --max-model-len engine-owned ( #49944 )
...
Co-authored-by: OpenAI Codex <codex@openai.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-27 14:50:08 +08:00
Sage
0f6cf7f628
[Rust Frontend] Extract request preparation from the inference path ( #49045 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-07-22 11:31:36 +00:00
Connor Carpenter and OpenAI Codex
c79ad3ae21
[Rust Frontend][gRPC] Add abort control RPC ( #49255 )
...
Co-authored-by: OpenAI Codex <codex@openai.com >
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
2026-07-22 11:31:01 +00:00
Bugen Zhao
eb44b3aaa4
[Rust][Benchmark] Use async HTTP clients ( #49295 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-21 16:53:57 +08:00
Bugen Zhao
8688a06d67
[Rust][Benchmark] Use tracing for logs ( #48937 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-21 05:18:13 +00:00
Salt Sato and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f25953cc59
[Bugfix][Rust Frontend] Handle zero-column logprobs payloads without panicking ( #49113 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Feathbow <feathbow@gmail.com >
2026-07-21 04:30:29 +00:00
