|
69db1c26b4
|
[CI/Build][Rust Frontend] Publish vllm-proto on crates.io (#56365)
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-11 16:28:52 +08:00 |
|
|
Bugen ZhaoandIsotr0py
|
b47b01cf33
|
[Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-10 01:15:06 -07:00 |
|
|
|
3735c2d5f5
|
[Security][Rust Frontend] Normalize HTTP method labels in metrics (#56058)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-10 10:54:40 +08:00 |
|
|
|
41cffa9af0
|
[Rust Frontend] Strongly type wire dtypes and multimodal modalities (#56174)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-10 02:11:22 +00:00 |
|
|
Bugen Zhao
|
285cbce6bd
|
[Rust Frontend] Resolve unified and split parser selections consistently (#56018)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 18:19:30 -07:00 |
|
|
|
c7e9816c6a
|
[Bugfix][Rust Frontend] Skip undefined token ids in decode and anchor them zero-width (#55240)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 09:55:04 +00:00 |
|
|
|
68dcc4fd86
|
[Rust Frontend] Make --max-model-len optional for the render server (#55328)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
|
2026-09-09 08:54:54 +00:00 |
|
|
Bugen ZhaoandCodex
|
fc6b6e1feb
|
[Rust Frontend] Correctly parse whitespace framing in model output (#55417)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-09 13:40:13 +10:00 |
|
|
Bugen ZhaoandCodex
|
050f29caf1
|
[Rust Frontend] Simplify reasoning parser initialization and test organization (#55411)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 10:57:15 +08:00 |
|
|
Connor Carpenter
|
761c5861e3
|
[Rust Frontend][gRPC] Preserve multimodal metadata for remote-prefill decode (#54814)
|
2026-09-05 02:37:53 +10:00 |
|
|
Ilya MarkovandBugen Zhao
|
ae2d1ca96f
|
[Rust Frontend] Record Mooncake/NIXL KV-connector metrics (#52755)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-04 06:21:54 +00:00 |
|
|
Bugen Zhao
|
d6bce42983
|
[Rust Frontend] Report reasoning tokens in chat completion usage (#54883)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-03 16:47:24 -07:00 |
|
|
|
c7e6e36fa9
|
[Rust Frontend] Add support for TLS in render server (#54999)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Richard <noreply@moonshot.cn>
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-03 21:56:44 +00:00 |
|
|
Will EatonandBugen Zhao
|
2db1c4dc31
|
[Rust Frontend] Support --lora-modules for static adapter loading (#54837)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-03 19:12:45 +00:00 |
|
|
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
21a2211040
|
[Rust Frontend] Use token-attributed text in reasoning and unified parsers (#54884)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-04 02:05:05 +08:00 |
|
|
Bugen Zhao
|
ad127d9a0f
|
[Perf][Rust Frontend] Coalesce decoded chunks per engine update (#55012)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-02 15:44:23 -07:00 |
|
|
Reid
|
60857baa53
|
[Bugfix][Rust Frontend][Renderer] Align DeepSeek V4 historical developer message handling (#54854)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-09-02 20:43:57 +00:00 |
|
|
Francisco Javier Arceo
|
c00091e026
|
[Frontend] Gate scale-out endpoints behind opt-in flag (#54579)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
|
2026-09-02 14:31:14 +08:00 |
|
|
Bugen Zhao
|
b911fe85c4
|
[Rust Frontend] Attribute decoded text to tokens (#52910)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-01 22:25:31 +00:00 |
|
|
Isotr0py
|
dc5cf437cf
|
[Rust Frontend] Enable Qwen4-exp multimodal support (#54813)
|
2026-09-01 20:23:55 +00:00 |
|
|
Bugen Zhao
|
3439bad37e
|
[Rust Frontend] Bound recursive argument parsers (#54303)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-01 12:45:50 -07:00 |
|
|
Jorge Menjivar
|
225aec4809
|
[Rust Frontend] Migrate to new tekken crate (#53056)
Signed-off-by: Jorge Menjivar <jorge@menjivar.ai>
|
2026-09-01 02:35:21 +00:00 |
|
|
|
22df3a34e0
|
[Perf][Rust Frontend] Count the tokenizer vocabulary once at construction (#54449)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-01 01:25:06 +00:00 |
|
|
|
d4329ba53d
|
[Bugfix][Rust Frontend] Fix adjacent DeepSeek V4 user content rendering (#53281)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-01 00:23:24 +00:00 |
|
|
Andreas KaratzasandOpenAI Codex
|
f9c7c6e090
|
[Rust Frontend][CI] Remove TCP port races from mock-engine tests (#54481)
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
|
2026-09-01 03:02:55 +08:00 |
|
|
wang.yuqiandCyrus Leung
|
eeb549a74d
|
[Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-08-31 10:13:50 +00:00 |
|
|
Connor Carpenter
|
7a67941c1d
|
[Rust Frontend][gRPC] Add audio and video media inputs (#53760)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
|
2026-08-30 11:31:37 +08:00 |
|
|
|
b2f685834a
|
[Hy4] support Hy4-preview model (#54160)
Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
|
2026-08-29 12:07:28 +08:00 |
|
|
|
4c6c9d569f
|
[Rust Frontend] Take the raw buffer in mm tensor lowering when possible (#53528)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: roachsinai <roachsinai@qq.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-29 01:05:14 +00:00 |
|
|
Pranav ThakurandBugen Zhao
|
085e9bb07e
|
[Rust Frontend] Add support for truncate_prompt_tokens and truncation_side (#48584)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: pranavthakur0-0 <pranavthakurdew@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-29 00:16:37 +00:00 |
|
|
|
74850f9f66
|
[Rust Frontend] Optimize SSE streaming hot path (#51321)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-28 20:35:38 +00:00 |
|
|
Bugen Zhao
|
11012d2a35
|
[Rust Frontend] Reduce copy in auxiliary frame resolution (#54148)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-28 09:01:17 -07:00 |
|
|
Connor CarpenterandBugen Zhao
|
3cc015acd2
|
[Rust Frontend][gRPC] Enforce LoRA path validation across transports (#53756)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-27 16:00:12 -07:00 |
|
|
Bugen ZhaoandNick Hill
|
9db222c6be
|
[Rust Frontend] Align OpenAI request and response edge cases (#53218)
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-27 19:23:38 +00:00 |
|
|
|
3fd72ffd7b
|
[Bugfix][Rust Frontend] Fix LogprobsTensors wire schema mismatch (#53939)
Signed-off-by: jungjiyu <libraryofjiyu@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-26 16:16:43 -07:00 |
|
|
Jiahao LiangandChauncey
|
59d7fc92ea
|
[Bugfix][DeepSeek V4] Handle trailing system messages in prompt rendering (#51262)
Signed-off-by: jiahaoliang <gzliangjiahao@gmail.com>
Signed-off-by: Jiahao Liang <gzliangjiahao@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-08-25 07:49:44 +00:00 |
|
|
|
1f9444a34f
|
[Rust Frontend][gRPC] Add LoRA lifecycle control (#52840)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-25 04:13:19 +00:00 |
|
|
Biswa Panda
|
9dba2c9b37
|
[Rust Frontend][RL]: report engine world size over gRPC (#53204)
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
|
2026-08-23 02:31:23 +00:00 |
|
|
Taneem Ibrahim
|
a014e35f38
|
[Pooling] Fix Rust pooling endpoint for benchmark (#53352)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-08-22 15:45:09 -04:00 |
|
|
Taneem Ibrahimandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
704f12aa1e
|
[Pooling] Report input throughput for batched requests (#53213)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-22 09:01:40 -04:00 |
|
|
Lucas Wilkinson
|
8bdc70ec7b
|
[6/N][KV-Cache Layout Refactor] Standardize KV cache layout (#51718)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2026-08-21 17:47:49 -07:00 |
|
|
Bugen ZhaoandOpenAI Codex
|
08b73d07cc
|
[Frontend] Prevent Kimi K3 reserved markers in response text (#52889)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-21 09:06:52 -07:00 |
|
|
Roy Wang
|
df344bf36f
|
[Rust][Benchmark] Load HF datasets from parquet shards via hf-hub, fixing truncated-cache sampling (#51570)
|
2026-08-21 05:07:43 +00:00 |
|
|
ReidandBugen Zhao
|
2adf4b9e2a
|
[Rust Frontend] Fix Kimi K3 reasoning_effort="none" handling (#53043)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-08-21 02:47:07 +00:00 |
|
|
Bugen Zhao
|
df3b3422b4
|
[Rust Frontend] Add HY3 unified parser and local XGrammar structural-tag builder (#53054)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-20 19:13:54 -07:00 |
|
|
|
83c5d59209
|
[Rust Frontend] Replace external protoc with pure Rust lib protox (#52892)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 18:43:51 -07:00 |
|
|
Bugen Zhao
|
f32b17b6d6
|
[Rust Frontend] Support --generation-config vllm (#53044)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-20 17:31:25 -07:00 |
|
|
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
|
|
727274a75b
|
[Rust Frontend] Fix Qwen parser auto-detection (#51169)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
|
2026-08-20 09:39:05 +00:00 |
|
|
|
963fcfa48c
|
[Rust][Benchmark] Align speed-bench CLI flags with Python and add flag parity test (#51592)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: esmeetu <jasonailu87@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-20 08:50:51 +00:00 |
|