3962042304
[Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift ( #46747 ) ( #46747 )
...
Signed-off-by: William Zhao <williamzh@meta.com >
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-12 12:06:57 +08:00
b64a2708b0
Bound generation inputs before expensive work ( #51447 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 22:03:50 +08:00
33584901ba
[Frontend] Add content_parts to /inference/v1/generate for raw multim… ( #51478 )
...
Signed-off-by: Allen Shen <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 10:00:12 +08:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f1e921b6d6
[Rust Frontend] Support dynamic tools from developer messages ( #51144 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-10 18:30:38 -07:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
d8c70f2243
[Rust Frontend] Upgrade MiniJinja to 2.22 & remove method lookup workaround ( #51235 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 00:23:24 +00:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fa722b9f01
[Rust Frontend][gRPC] Add explicit data-parallel rank routing ( #51178 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-10 15:05:24 -07:00
Sage
865781e627
[Rust Frontend] Add standalone Rust renderer ( #50289 )
2026-08-06 11:23:26 +00:00
Chauncey
22170354d8
[Feature] Parse request priority from HTTP header ( #51089 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-05 21:10:29 -07:00
Sage and Bugen Zhao
5bf32605a9
[Rust Frontend] Deduplicate request preprocessing for /tokenize ( #50448 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-05 08:27:35 +00:00
Bugen Zhao
7743486190
[Frontend] DeepSeek V4 0731 reasoning effort prompts & mappings ( #50580 )
2026-08-04 08:51:32 -07:00
Connor Carpenter and Bugen Zhao
e98a8774cb
[Rust Frontend][gRPC] Add multimodal image inference ( #50368 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-04 09:34:31 +00:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
0b1c151cbc
[Rust Frontend] Align tool rendering for Kimi K3 ( #50540 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-04 08:03:46 +00:00
Reid
72cd5424da
[Rust][Benchmark] Preserve UTF-8 across benchmark stream chunks ( #50868 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-08-04 06:15:02 +00:00
Karen Chung
f57123aa2d
feat(frontend): session id plumbing into requests ( #48048 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-08-03 22:18:58 +00:00
zcxGGmu and zq
c4e9f09de7
[Bugfix][Frontend] Reject empty gRPC stop strings ( #50746 )
...
Signed-off-by: zq <zhouquan1511@163.com >
Co-authored-by: zq <zhouquan1511@163.com >
2026-08-03 20:50:15 +00:00
Reid
62195e9784
[Rust][Benchmark] Prevent invalid token IDs in random benchmarks ( #50058 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-08-01 03:20:29 +00:00
Bugen Zhao
ab98034d4c
[Frontend][Bugfix] Use default tool call IDs for Kimi K3 for conversation-level uniqueness ( #50420 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-31 10:02:17 +08:00
Samuel Laferriere and Claude
d6938b7408
[Bugfix][Rust Frontend] Select earliest-completing stop string ( #50200 )
...
Signed-off-by: Samuel Laferriere <9342524+samlaf@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-31 00:33:19 +00:00
Bugen Zhao and OpenAI Codex
e9096fcb12
[Rust Frontend] Improve startup failure and readiness logs ( #50406 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-30 14:18:17 -07:00
Bugen Zhao
629a938a92
[Frontend] Preserve bare Inkling text in Python and Rust parsers ( #50403 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-30 12:25:10 -07:00
Connor Carpenter and Nick Hill
48fc2e2797
feat(grpc): add KV event source discovery ( #50033 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-29 13:46:40 -07:00
Reid and Bugen Zhao
242c591d5a
[Rust Frontend] Send multimodal tensors in auxiliary frames ( #49341 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-29 12:21:09 +00:00
Maria Guevara
c44e191b01
[Rust Frontend] Add --limit-mm-per-prompt support ( #49604 )
...
Signed-off-by: Maria Guevara <kawaiiplush14@gmail.com >
2026-07-29 17:21:41 +08:00
Bugen Zhao
2899dca843
[Model] Add Kimi K3 support: Rust frontend [1/2] ( #50104 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-28 14:31:13 -07:00
Bugen Zhao
d552a68645
[Rust Frontend] Extract shared tracing setup logic into vllm-tracing ( #50129 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-28 19:31:15 +00:00
Reid
912d6b619d
[Rust Frontend] Align sampling validation with Python ( #47494 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-28 12:02:00 +00:00
Reid
bf9f23003c
[Rust Frontend] Fix finish reason for named tool choices ( #49496 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-28 10:53:18 +00:00
Bugen Zhao and OpenAI Codex
5ed3faa43d
[Rust Frontend] Add ordinary-text tokenizer encoding ( #49992 )
...
Co-authored-by: OpenAI Codex <noreply@openai.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-28 15:46:51 +08:00
Connor Carpenter and Nick Hill
e3c2fc3b3c
[Rust Frontend][gRPC] Add server and model discovery ( #49491 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-27 09:53:27 -07:00
Nick Hill and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
59a6b0411d
[Core] Fix internal LB load-balancing ( #49204 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-27 14:00:46 +00:00
Bugen Zhao and OpenAI Codex
fd9d2ede6f
[Rust Frontend] Keep --max-model-len engine-owned ( #49944 )
...
Co-authored-by: OpenAI Codex <codex@openai.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-27 14:50:08 +08:00
Sage
0f6cf7f628
[Rust Frontend] Extract request preparation from the inference path ( #49045 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-07-22 11:31:36 +00:00
Connor Carpenter and OpenAI Codex
c79ad3ae21
[Rust Frontend][gRPC] Add abort control RPC ( #49255 )
...
Co-authored-by: OpenAI Codex <codex@openai.com >
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
2026-07-22 11:31:01 +00:00
Bugen Zhao
eb44b3aaa4
[Rust][Benchmark] Use async HTTP clients ( #49295 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-21 16:53:57 +08:00
Bugen Zhao
8688a06d67
[Rust][Benchmark] Use tracing for logs ( #48937 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-21 05:18:13 +00:00
Salt Sato and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f25953cc59
[Bugfix][Rust Frontend] Handle zero-column logprobs payloads without panicking ( #49113 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Feathbow <feathbow@gmail.com >
2026-07-21 04:30:29 +00:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1940c8441e
[Rust Frontend][gRPC] Add engine-aware health reporting ( #48992 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
2026-07-21 10:54:39 +08:00
Bugen Zhao
e2d7adeb64
[Rust Frontend] Bump xgrammar-structural-tag and enable local extension ( #49161 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-20 16:22:24 +01:00
Salt Sato
d835ad572c
[Bugfix][Rust Frontend] Map missing prompt logprobs for single-token prompts in chat and raw generate ( #49111 )
...
Signed-off-by: Feathbow <feathbow@gmail.com >
2026-07-20 10:00:06 +00:00
Reid
818cf61e91
[Rust Frontend] Fix macro-based content format detection ( #49042 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-20 09:39:13 +00:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
c01618fdc8
[Rust][Benchmark] Integrate vllm-bench to vllm-rs & vllm CLI ( #48930 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-20 09:31:25 +00:00
Sage and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f1f1259692
[Rust Frontend] Use zero-copy slicing for multimodal tensors ( #48781 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-07-20 16:28:25 +08:00
yzong-rh
a287eb163f
[Front-end] [Messages] Populate num_cache_creation_tokens ( #48535 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-07-18 13:04:35 -04:00
Roy Wang and Bugen Zhao
9354f22204
[Rust][Benchmark] Port in vllm-bench ( #48107 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: esmeetu <jasonailu87@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-17 14:25:29 +08:00
Reid
8c3393f373
[Bugfix][Rust Frontend] Limit chat top_logprobs in responses ( #48134 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-16 09:41:44 +00:00
Chao-Ju Chen and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
ea1d65fe6d
[Rust Frontend] Add Seed-OSS tool parser ( #47741 )
...
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: RickyChen / 陳昭儒 <ricky.chen@infinirc.com >
2026-07-16 17:28:02 +08:00
Reid and Bugen Zhao
f44f3d6f79
[Rust Frontend] Wait for mock engine endpoints before ZMQ connect ( #47965 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-16 09:20:31 +00:00
JinYan Su and Bugen Zhao
cc706b05a5
[Bugfix][Rust Frontend] Detokenizer: avoid leaking prompt on zero-generated-token completions ( #47707 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: xiaguan <751080330@qq.com >
2026-07-16 09:11:21 +00:00
Reid
dc9f845ddc
[Rust Frontend] Fix mock engine test shutdown race ( #48738 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-16 07:27:43 +00:00
6570c9800c
[Model] Add Inkling model support [1/N] ( #48799 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-15 23:40:07 -07:00
