 Bugen ZhaoandCodex
|
81d7293c21
|
[Rust Frontend] Construct model-owned vision processors through specs (#58109)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-22 10:01:53 +00:00 |
|
|
Bugen Zhao
|
a6c47fbbf4
|
[Rust Frontend] Separate multimodal instrumentation from request timing (#58084)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-22 06:57:56 +00:00 |
|
|
Bugen ZhaoandCodex
|
496c6472cb
|
[Rust Frontend] Build full-output grammars from initialized reasoning parsers (#57340)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-22 06:11:55 +00:00 |
|
|
Bugen ZhaoandClaude Fable 5.1
|
d50723df04
|
[Rust Frontend] Introduce parser-owned output grammar interfaces (#55269)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-22 05:35:07 +00:00 |
|
|
|
f84325c48c
|
[Frontend][Rust] Add mm-processor benchmark for Rust frontend (#51922)
Signed-off-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Signed-off-by: gangula-karthik <gkarthik923@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Karthik Gangula <gangula-karthik@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-22 05:08:35 +00:00 |
|
|
Bugen ZhaoandCodex
|
ff3c9cba3f
|
[Rust Frontend] Add MiMo V2.5 parser support (#57933)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-22 04:27:09 +00:00 |
|
|
Karen Chung
|
986e217158
|
[Feature] Add first-class KV hints request envelope for programmatic KV management (#53423)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-09-21 09:56:11 -07:00 |
|
|
|
10e6a7f210
|
[Frontend] Expose per-request spec decode metrics in generate API (#43310)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-20 12:21:33 +00:00 |
|
|
|
90531daf6d
|
[Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-09-18 18:11:05 -07:00 |
|
|
|
c58532c86b
|
[Frontend] Upgrade XGrammar to 0.2.7 and Rust structural tags to 0.3.0 (#57272)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 11:44:05 +00:00 |
|
|
Biswa PandaandBugen Zhao
|
95d723700b
|
[Rust Frontend] Return sampling masks over gRPC (#56777)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
|
2026-09-18 08:26:17 +00:00 |
|
|
|
39e33db7f3
|
[Frontend] Fix the parsing of missing string= in DeepSeek V4 (#56271)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
|
2026-09-18 05:11:24 +00:00 |
|
|
|
dee37d8911
|
[Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 04:32:39 +00:00 |
|
|
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
|
aoshen02
|
b978df3299
|
Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302)
|
2026-09-16 23:31:59 -07:00 |
|
|
aoshen02andClaude Fable 5.1
|
8538017f4b
|
[Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-17 05:58:56 +00:00 |
|
|
Alec
|
d2a2a82954
|
[Rust Frontend] Bump vllm-proto to 0.3.0 (#57233)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
|
2026-09-17 03:41:55 +00:00 |
|
|
AlecandCodex
|
28ec03bc6e
|
[Bugfix][Rust Frontend] Accept appended EngineCoreOutput fields (#56533)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
|
2026-09-17 01:54:43 +00:00 |
|
|
Bugen Zhao
|
e52da038d1
|
[Rust Frontend] Normalize native renderer reasoning controls (#56998)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 18:15:05 -07:00 |
|
|
aoshen02
|
0983aef8da
|
[Frontend] Return prompt metadata from /inference/v1/generate (#53187)
|
2026-09-16 15:29:07 -07:00 |
|
|
AlecandCodex
|
9639cbde04
|
[Rust Frontend] Expose local DP size in gRPC Control metadata (#57116)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-16 11:28:08 -07:00 |
|
|
|
f37c550bf6
|
[Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 08:29:43 +00:00 |
|
|
IslamandBugen Zhao
|
c16bb6068f
|
[Bugfix][Rust Frontend] Tolerate NaN-corrupted logprobs in engine-core (#51026)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Islam <islam.almersawi@openinnovation.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 03:10:58 +00:00 |
|
|
|
fdfcbbcac7
|
[CI/Build][Rust Frontend] Retire Buf schema publishing (#57046)
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
|
2026-09-16 03:02:25 +00:00 |
|
|
Bugen Zhao
|
bfd713bf8b
|
[Rust Frontend] Support HF config overrides --hf-overrides (#56931)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 02:52:55 +00:00 |
|
|
ReidandBugen Zhao
|
0673de9800
|
[Rust Frontend][Metrics] Add per-request preemption histogram (#57033)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-09-16 02:51:17 +00:00 |
|
|
Bugen ZhaoandCodex
|
78eaaad2c7
|
[Rust Frontend] Add iteration token histogram (#56990)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-15 12:12:15 +00:00 |
|
|
Yeonwoo SungandBugen Zhao
|
142020c6c2
|
[Bugfix][Rust Frontend] Honor add_generation_prompt in DeepSeek V3.2/V4/V4.1 renderers (#56593)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: YeonwooSung <neos960518@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-15 10:03:26 +00:00 |
|
|
Kevin H. LuuandOpenAI Codex
|
772e4f05fa
|
[Bugfix][Rust] Fix HF multi-turn dataset integration (#56915)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-14 18:04:46 -07:00 |
|
|
|
f0a61bd438
|
[Rust][Benchmark] Support HF ShareGPT datasets in multi-turn mode (#51104)
Signed-off-by: xiaguan <751080330@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 16:48:23 -07:00 |
|
|
Kevin H. LuuandSherlock
|
d320a21431
|
[Bugfix][Rust Frontend] Restore prost test dependency (#56866)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
|
2026-09-14 14:37:44 -07:00 |
|
|
Nicolò LucchesiandCodex
|
1713a09491
|
[Rust Frontend] Forward per-request watermarking controls (#56338)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-14 23:17:02 +08:00 |
|
|
|
7b1ea3f524
|
[Bugfix][Rust Frontend][Multimodal] Align DeepSeek V4.1 and Kimi K3 media with rendered placeholders (#56366)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-14 10:37:59 +00:00 |
|
|
Martin Hickey
|
934b1fcbcb
|
[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-14 10:02:14 +00:00 |
|
|
AlecandOpenAI Codex
|
e6b4e47d2d
|
[Bugfix][Rust Frontend] Preserve selected-token logprob mode (#56406)
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
|
2026-09-14 07:05:33 +00:00 |
|
|
|
7632f767f0
|
[Rust Frontend] Support HTTP RL weight synchronization (#56567)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-14 03:37:04 +00:00 |
|
|
Clinton ThomasandLucas Bourtoule
|
22f6e4eccb
|
[Frontend][Rust] Reject empty structured-output values (#54821)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
|
2026-09-12 19:12:25 +04:00 |
|
|
Bugen Zhao
|
eed1f3d0c6
|
[Rust Frontend] Support generation blocks in HF chat template (#56378)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-12 04:56:55 +00:00 |
|
|
Bugen ZhaoandCodex
|
485421b1c3
|
[Rust Frontend] Honor HF revisions, offline mode, and cache directory (#56386)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-12 03:39:17 +00:00 |
|
|
|
31759ccb6d
|
[Rust Frontend][Multimodal] Accept preprocessed multimodal gRPC features (#55047)
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-12 02:29:49 +00:00 |
|
|
AlecandCodex
|
988d9b6777
|
[Rust Frontend][gRPC] Surface engine generation errors (#56405)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-11 15:31:55 -07:00 |
|
|
|
79c137c6ad
|
[Bugfix][Rust Frontend][Renderer] Align DeepSeek tool-call arguments with deepseek-recipe (#56260)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: JinYan Su <751080330@qq.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-11 10:19:55 +00:00 |
|
|
|
69db1c26b4
|
[CI/Build][Rust Frontend] Publish vllm-proto on crates.io (#56365)
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-11 16:28:52 +08:00 |
|
|
Bugen ZhaoandIsotr0py
|
b47b01cf33
|
[Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-10 01:15:06 -07:00 |
|
|
|
3735c2d5f5
|
[Security][Rust Frontend] Normalize HTTP method labels in metrics (#56058)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-10 10:54:40 +08:00 |
|
|
|
41cffa9af0
|
[Rust Frontend] Strongly type wire dtypes and multimodal modalities (#56174)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-10 02:11:22 +00:00 |
|
|
Bugen Zhao
|
285cbce6bd
|
[Rust Frontend] Resolve unified and split parser selections consistently (#56018)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 18:19:30 -07:00 |
|
|
|
c7e9816c6a
|
[Bugfix][Rust Frontend] Skip undefined token ids in decode and anchor them zero-width (#55240)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 09:55:04 +00:00 |
|
|
|
68dcc4fd86
|
[Rust Frontend] Make --max-model-len optional for the render server (#55328)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
|
2026-09-09 08:54:54 +00:00 |
|
|
Bugen ZhaoandCodex
|
fc6b6e1feb
|
[Rust Frontend] Correctly parse whitespace framing in model output (#55417)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-09 13:40:13 +10:00 |
|