 Karen Chung
|
986e217158
|
[Feature] Add first-class KV hints request envelope for programmatic KV management (#53423)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-09-21 09:56:11 -07:00 |
|
|
|
10e6a7f210
|
[Frontend] Expose per-request spec decode metrics in generate API (#43310)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-20 12:21:33 +00:00 |
|
|
|
90531daf6d
|
[Core][Frontend] Bind KV-event publishers at port 0 and expose the bound endpoints (#55844)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: mukeshbhandarkar <shell00035@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-09-18 18:11:05 -07:00 |
|
|
|
c58532c86b
|
[Frontend] Upgrade XGrammar to 0.2.7 and Rust structural tags to 0.3.0 (#57272)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 11:44:05 +00:00 |
|
|
Biswa PandaandBugen Zhao
|
95d723700b
|
[Rust Frontend] Return sampling masks over gRPC (#56777)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
|
2026-09-18 08:26:17 +00:00 |
|
|
|
39e33db7f3
|
[Frontend] Fix the parsing of missing string= in DeepSeek V4 (#56271)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
|
2026-09-18 05:11:24 +00:00 |
|
|
|
dee37d8911
|
[Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 04:32:39 +00:00 |
|
|
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
|
aoshen02
|
b978df3299
|
Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302)
|
2026-09-16 23:31:59 -07:00 |
|
|
aoshen02andClaude Fable 5.1
|
8538017f4b
|
[Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-17 05:58:56 +00:00 |
|
|
Alec
|
d2a2a82954
|
[Rust Frontend] Bump vllm-proto to 0.3.0 (#57233)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
|
2026-09-17 03:41:55 +00:00 |
|
|
AlecandCodex
|
28ec03bc6e
|
[Bugfix][Rust Frontend] Accept appended EngineCoreOutput fields (#56533)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
|
2026-09-17 01:54:43 +00:00 |
|
|
Bugen Zhao
|
e52da038d1
|
[Rust Frontend] Normalize native renderer reasoning controls (#56998)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 18:15:05 -07:00 |
|
|
aoshen02
|
0983aef8da
|
[Frontend] Return prompt metadata from /inference/v1/generate (#53187)
|
2026-09-16 15:29:07 -07:00 |
|
|
AlecandCodex
|
9639cbde04
|
[Rust Frontend] Expose local DP size in gRPC Control metadata (#57116)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-16 11:28:08 -07:00 |
|
|
|
f37c550bf6
|
[Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 08:29:43 +00:00 |
|
|
IslamandBugen Zhao
|
c16bb6068f
|
[Bugfix][Rust Frontend] Tolerate NaN-corrupted logprobs in engine-core (#51026)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Islam <islam.almersawi@openinnovation.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 03:10:58 +00:00 |
|
|
|
fdfcbbcac7
|
[CI/Build][Rust Frontend] Retire Buf schema publishing (#57046)
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
|
2026-09-16 03:02:25 +00:00 |
|
|
Bugen Zhao
|
bfd713bf8b
|
[Rust Frontend] Support HF config overrides --hf-overrides (#56931)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 02:52:55 +00:00 |
|
|
ReidandBugen Zhao
|
0673de9800
|
[Rust Frontend][Metrics] Add per-request preemption histogram (#57033)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-09-16 02:51:17 +00:00 |
|
|
Bugen ZhaoandCodex
|
78eaaad2c7
|
[Rust Frontend] Add iteration token histogram (#56990)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-15 12:12:15 +00:00 |
|
|
Yeonwoo SungandBugen Zhao
|
142020c6c2
|
[Bugfix][Rust Frontend] Honor add_generation_prompt in DeepSeek V3.2/V4/V4.1 renderers (#56593)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: YeonwooSung <neos960518@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-15 10:03:26 +00:00 |
|
|
Kevin H. LuuandOpenAI Codex
|
772e4f05fa
|
[Bugfix][Rust] Fix HF multi-turn dataset integration (#56915)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-14 18:04:46 -07:00 |
|
|
|
f0a61bd438
|
[Rust][Benchmark] Support HF ShareGPT datasets in multi-turn mode (#51104)
Signed-off-by: xiaguan <751080330@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-14 16:48:23 -07:00 |
|
|
Kevin H. LuuandSherlock
|
d320a21431
|
[Bugfix][Rust Frontend] Restore prost test dependency (#56866)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Co-authored-by: Sherlock <sherlock@raft.ai>
|
2026-09-14 14:37:44 -07:00 |
|
|
Nicolò LucchesiandCodex
|
1713a09491
|
[Rust Frontend] Forward per-request watermarking controls (#56338)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-14 23:17:02 +08:00 |
|
|
|
7b1ea3f524
|
[Bugfix][Rust Frontend][Multimodal] Align DeepSeek V4.1 and Kimi K3 media with rendered placeholders (#56366)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-14 10:37:59 +00:00 |
|
|
Martin Hickey
|
934b1fcbcb
|
[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-14 10:02:14 +00:00 |
|
|
AlecandOpenAI Codex
|
e6b4e47d2d
|
[Bugfix][Rust Frontend] Preserve selected-token logprob mode (#56406)
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
|
2026-09-14 07:05:33 +00:00 |
|
|
|
7632f767f0
|
[Rust Frontend] Support HTTP RL weight synchronization (#56567)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Wang Zupeng <zupenwang@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-14 03:37:04 +00:00 |
|
|
Clinton ThomasandLucas Bourtoule
|
22f6e4eccb
|
[Frontend][Rust] Reject empty structured-output values (#54821)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
|
2026-09-12 19:12:25 +04:00 |
|
|
Bugen Zhao
|
eed1f3d0c6
|
[Rust Frontend] Support generation blocks in HF chat template (#56378)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-12 04:56:55 +00:00 |
|
|
Bugen ZhaoandCodex
|
485421b1c3
|
[Rust Frontend] Honor HF revisions, offline mode, and cache directory (#56386)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-12 03:39:17 +00:00 |
|
|
|
31759ccb6d
|
[Rust Frontend][Multimodal] Accept preprocessed multimodal gRPC features (#55047)
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-12 02:29:49 +00:00 |
|
|
AlecandCodex
|
988d9b6777
|
[Rust Frontend][gRPC] Surface engine generation errors (#56405)
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-11 15:31:55 -07:00 |
|
|
|
79c137c6ad
|
[Bugfix][Rust Frontend][Renderer] Align DeepSeek tool-call arguments with deepseek-recipe (#56260)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: JinYan Su <751080330@qq.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-11 10:19:55 +00:00 |
|
|
|
69db1c26b4
|
[CI/Build][Rust Frontend] Publish vllm-proto on crates.io (#56365)
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-11 16:28:52 +08:00 |
|
|
Bugen ZhaoandIsotr0py
|
b47b01cf33
|
[Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-09-10 01:15:06 -07:00 |
|
|
|
3735c2d5f5
|
[Security][Rust Frontend] Normalize HTTP method labels in metrics (#56058)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-10 10:54:40 +08:00 |
|
|
|
41cffa9af0
|
[Rust Frontend] Strongly type wire dtypes and multimodal modalities (#56174)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-10 02:11:22 +00:00 |
|
|
Bugen Zhao
|
285cbce6bd
|
[Rust Frontend] Resolve unified and split parser selections consistently (#56018)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 18:19:30 -07:00 |
|
|
|
c7e9816c6a
|
[Bugfix][Rust Frontend] Skip undefined token ids in decode and anchor them zero-width (#55240)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 09:55:04 +00:00 |
|
|
|
68dcc4fd86
|
[Rust Frontend] Make --max-model-len optional for the render server (#55328)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
|
2026-09-09 08:54:54 +00:00 |
|
|
Bugen ZhaoandCodex
|
fc6b6e1feb
|
[Rust Frontend] Correctly parse whitespace framing in model output (#55417)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-09 13:40:13 +10:00 |
|
|
Bugen ZhaoandCodex
|
050f29caf1
|
[Rust Frontend] Simplify reasoning parser initialization and test organization (#55411)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-09 10:57:15 +08:00 |
|
|
Connor Carpenter
|
761c5861e3
|
[Rust Frontend][gRPC] Preserve multimodal metadata for remote-prefill decode (#54814)
|
2026-09-05 02:37:53 +10:00 |
|
|
Ilya MarkovandBugen Zhao
|
ae2d1ca96f
|
[Rust Frontend] Record Mooncake/NIXL KV-connector metrics (#52755)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-04 06:21:54 +00:00 |
|
|
Bugen Zhao
|
d6bce42983
|
[Rust Frontend] Report reasoning tokens in chat completion usage (#54883)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-03 16:47:24 -07:00 |
|
|
|
c7e6e36fa9
|
[Rust Frontend] Add support for TLS in render server (#54999)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Richard <noreply@moonshot.cn>
Signed-off-by: Wen Zhou <wenzhou@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-03 21:56:44 +00:00 |
|
|
Will EatonandBugen Zhao
|
2db1c4dc31
|
[Rust Frontend] Support --lora-modules for static adapter loading (#54837)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-03 19:12:45 +00:00 |
|