Martin Hickey
934b1fcbcb
[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out ( #55176 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-09-14 10:02:14 +00:00
Alec and OpenAI Codex
e6b4e47d2d
[Bugfix][Rust Frontend] Preserve selected-token logprob mode ( #56406 )
...
Co-authored-by: OpenAI Codex <noreply@openai.com >
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
2026-09-14 07:05:33 +00:00
7632f767f0
[Rust Frontend] Support HTTP RL weight synchronization ( #56567 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Codex <noreply@openai.com >
Signed-off-by: Wang Zupeng <zupenwang@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-14 03:37:04 +00:00
Clinton Thomas and Lucas Bourtoule
22f6e4eccb
[Frontend][Rust] Reject empty structured-output values ( #54821 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
2026-09-12 19:12:25 +04:00
Bugen Zhao and Codex
485421b1c3
[Rust Frontend] Honor HF revisions, offline mode, and cache directory ( #56386 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-12 03:39:17 +00:00
31759ccb6d
[Rust Frontend][Multimodal] Accept preprocessed multimodal gRPC features ( #55047 )
...
Signed-off-by: Biswa Panda <biswa.panda@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-12 02:29:49 +00:00
Alec and Codex
988d9b6777
[Rust Frontend][gRPC] Surface engine generation errors ( #56405 )
...
Signed-off-by: Alec Flowers <aflowers@nvidia.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-11 15:31:55 -07:00
69db1c26b4
[CI/Build][Rust Frontend] Publish vllm-proto on crates.io ( #56365 )
...
Co-authored-by: OpenAI Codex <noreply@openai.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-11 16:28:52 +08:00
Bugen Zhao and Isotr0py
b47b01cf33
[Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends ( #56208 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 01:15:06 -07:00
3735c2d5f5
[Security][Rust Frontend] Normalize HTTP method labels in metrics ( #56058 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-10 10:54:40 +08:00
41cffa9af0
[Rust Frontend] Strongly type wire dtypes and multimodal modalities ( #56174 )
...
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-10 02:11:22 +00:00
68dcc4fd86
[Rust Frontend] Make --max-model-len optional for the render server ( #55328 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Wen Zhou <wenzhou@redhat.com >
2026-09-09 08:54:54 +00:00
Connor Carpenter
761c5861e3
[Rust Frontend][gRPC] Preserve multimodal metadata for remote-prefill decode ( #54814 )
2026-09-05 02:37:53 +10:00
Bugen Zhao
d6bce42983
[Rust Frontend] Report reasoning tokens in chat completion usage ( #54883 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-03 16:47:24 -07:00
c7e6e36fa9
[Rust Frontend] Add support for TLS in render server ( #54999 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Richard <noreply@moonshot.cn >
Signed-off-by: Wen Zhou <wenzhou@redhat.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-03 21:56:44 +00:00
Will Eaton and Bugen Zhao
2db1c4dc31
[Rust Frontend] Support --lora-modules for static adapter loading ( #54837 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Will Eaton <weaton@redhat.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-03 19:12:45 +00:00
Francisco Javier Arceo
c00091e026
[Frontend] Gate scale-out endpoints behind opt-in flag ( #54579 )
...
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com >
2026-09-02 14:31:14 +08:00
Bugen Zhao
b911fe85c4
[Rust Frontend] Attribute decoded text to tokens ( #52910 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-01 22:25:31 +00:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
Connor Carpenter
7a67941c1d
[Rust Frontend][gRPC] Add audio and video media inputs ( #53760 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
2026-08-30 11:31:37 +08:00
Pranav Thakur and Bugen Zhao
085e9bb07e
[Rust Frontend] Add support for truncate_prompt_tokens and truncation_side ( #48584 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: pranavthakur0-0 <pranavthakurdew@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-29 00:16:37 +00:00
74850f9f66
[Rust Frontend] Optimize SSE streaming hot path ( #51321 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Signed-off-by: reidliu41 <reid201711@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-28 20:35:38 +00:00
Connor Carpenter and Bugen Zhao
3cc015acd2
[Rust Frontend][gRPC] Enforce LoRA path validation across transports ( #53756 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-08-27 16:00:12 -07:00
Bugen Zhao and Nick Hill
9db222c6be
[Rust Frontend] Align OpenAI request and response edge cases ( #53218 )
...
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-27 19:23:38 +00:00
1f9444a34f
[Rust Frontend][gRPC] Add LoRA lifecycle control ( #52840 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-25 04:13:19 +00:00
Biswa Panda
9dba2c9b37
[Rust Frontend][RL]: report engine world size over gRPC ( #53204 )
...
Signed-off-by: Biswa Panda <biswa.panda@gmail.com >
2026-08-23 02:31:23 +00:00
83c5d59209
[Rust Frontend] Replace external protoc with pure Rust lib protox ( #52892 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-20 18:43:51 -07:00
Bugen Zhao
f32b17b6d6
[Rust Frontend] Support --generation-config vllm ( #53044 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-20 17:31:25 -07:00
Quentin Gallouédec and Bugen Zhao
16cfe728d8
[Bugfix][Rust Frontend] Reject n > 1 in the /inference/v1/generate route ( #52844 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-20 07:41:06 +00:00
Bugen Zhao and Nick Hill
41f179b57a
[Rust Frontend] Simplify data-parallel size ownership ( #52575 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-18 09:06:05 +00:00
c89d692cb5
[Build] Propagate vLLM version to Rust binaries ( #52593 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-08-17 23:37:37 -07:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f08a95f8d8
[Rust Frontend][gRPC] Advertise LoRA capabilities ( #52031 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-17 13:27:56 -07:00
Biswa Panda
42156466db
[Rust Frontend][gRPC] Preserve skip_special_tokens decoding option ( #52384 )
...
Signed-off-by: Biswa Panda <biswa.panda@gmail.com >
2026-08-15 08:54:58 +00:00
Connor Carpenter and Nick Hill
f473870ecf
[Rust Frontend][gRPC] Add RL lifecycle control ( #51316 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-14 11:32:39 -07:00
3962042304
[Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift ( #46747 ) ( #46747 )
...
Signed-off-by: William Zhao <williamzh@meta.com >
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
Co-authored-by: Claude <noreply@anthropic.com >
2026-08-12 12:06:57 +08:00
b64a2708b0
Bound generation inputs before expensive work ( #51447 )
...
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com >
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 22:03:50 +08:00
33584901ba
[Frontend] Add content_parts to /inference/v1/generate for raw multim… ( #51478 )
...
Signed-off-by: Allen Shen <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 10:00:12 +08:00
Bugen Zhao and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
f1e921b6d6
[Rust Frontend] Support dynamic tools from developer messages ( #51144 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-10 18:30:38 -07:00
Connor Carpenter and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
fa722b9f01
[Rust Frontend][gRPC] Add explicit data-parallel rank routing ( #51178 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-10 15:05:24 -07:00
Sage
865781e627
[Rust Frontend] Add standalone Rust renderer ( #50289 )
2026-08-06 11:23:26 +00:00
Chauncey
22170354d8
[Feature] Parse request priority from HTTP header ( #51089 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-05 21:10:29 -07:00
Sage and Bugen Zhao
5bf32605a9
[Rust Frontend] Deduplicate request preprocessing for /tokenize ( #50448 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-05 08:27:35 +00:00
Connor Carpenter and Bugen Zhao
e98a8774cb
[Rust Frontend][gRPC] Add multimodal image inference ( #50368 )
...
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-08-04 09:34:31 +00:00
Karen Chung
f57123aa2d
feat(frontend): session id plumbing into requests ( #48048 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-08-03 22:18:58 +00:00
zcxGGmu and zq
c4e9f09de7
[Bugfix][Frontend] Reject empty gRPC stop strings ( #50746 )
...
Signed-off-by: zq <zhouquan1511@163.com >
Co-authored-by: zq <zhouquan1511@163.com >
2026-08-03 20:50:15 +00:00
Bugen Zhao and OpenAI Codex
e9096fcb12
[Rust Frontend] Improve startup failure and readiness logs ( #50406 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-30 14:18:17 -07:00
Connor Carpenter and Nick Hill
48fc2e2797
feat(grpc): add KV event source discovery ( #50033 )
...
Signed-off-by: Connor Carpenter <connorc@nvidia.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-29 13:46:40 -07:00
Maria Guevara
c44e191b01
[Rust Frontend] Add --limit-mm-per-prompt support ( #49604 )
...
Signed-off-by: Maria Guevara <kawaiiplush14@gmail.com >
2026-07-29 17:21:41 +08:00
Bugen Zhao
2899dca843
[Model] Add Kimi K3 support: Rust frontend [1/2] ( #50104 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-28 14:31:13 -07:00
Reid
912d6b619d
[Rust Frontend] Align sampling validation with Python ( #47494 )
...
Signed-off-by: reidliu41 <reid201711@gmail.com >
2026-07-28 12:02:00 +00:00
Martin Hickey