 Bugen ZhaoandClaude Fable 5.1
|
d50723df04
|
[Rust Frontend] Introduce parser-owned output grammar interfaces (#55269)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-22 05:35:07 +00:00 |
|
|
Karen Chung
|
986e217158
|
[Feature] Add first-class KV hints request envelope for programmatic KV management (#53423)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-09-21 09:56:11 -07:00 |
|
|
|
10e6a7f210
|
[Frontend] Expose per-request spec decode metrics in generate API (#43310)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-20 12:21:33 +00:00 |
|
|
Biswa PandaandBugen Zhao
|
95d723700b
|
[Rust Frontend] Return sampling masks over gRPC (#56777)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
|
2026-09-18 08:26:17 +00:00 |
|
|
|
092bdd6d57
|
[Frontend][Core] Add release_kv_cache_memory() API (#44890)
Signed-off-by: Dakai An <dakaian108@gmail.com>
Signed-off-by: AlanFokCo <alanfok2868@gmail.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <77474977+andakai@users.noreply.github.com>
Co-authored-by: AlanFokCo <alanfok2868@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-18 07:12:12 +08:00 |
|
|
AlecandCodex
|
28ec03bc6e
|
[Bugfix][Rust Frontend] Accept appended EngineCoreOutput fields (#56533)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
|
2026-09-17 01:54:43 +00:00 |
|
|
|
f37c550bf6
|
[Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 08:29:43 +00:00 |
|
|
IslamandBugen Zhao
|
c16bb6068f
|
[Bugfix][Rust Frontend] Tolerate NaN-corrupted logprobs in engine-core (#51026)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Islam <islam.almersawi@openinnovation.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 03:10:58 +00:00 |
|
|
Bugen ZhaoandCodex
|
78eaaad2c7
|
[Rust Frontend] Add iteration token histogram (#56990)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-15 12:12:15 +00:00 |
|
|
Nicolò LucchesiandCodex
|
1713a09491
|
[Rust Frontend] Forward per-request watermarking controls (#56338)
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-14 23:17:02 +08:00 |
|
|
Clinton ThomasandLucas Bourtoule
|
22f6e4eccb
|
[Frontend][Rust] Reject empty structured-output values (#54821)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
|
2026-09-12 19:12:25 +04:00 |
|
|
|
41cffa9af0
|
[Rust Frontend] Strongly type wire dtypes and multimodal modalities (#56174)
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-10 02:11:22 +00:00 |
|
|
Ilya MarkovandBugen Zhao
|
ae2d1ca96f
|
[Rust Frontend] Record Mooncake/NIXL KV-connector metrics (#52755)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Markov Ilya <markovilya197@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-04 06:21:54 +00:00 |
|
|
Will EatonandBugen Zhao
|
2db1c4dc31
|
[Rust Frontend] Support --lora-modules for static adapter loading (#54837)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Will Eaton <weaton@redhat.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-03 19:12:45 +00:00 |
|
|
Bugen Zhao
|
11012d2a35
|
[Rust Frontend] Reduce copy in auxiliary frame resolution (#54148)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-28 09:01:17 -07:00 |
|
|
|
3fd72ffd7b
|
[Bugfix][Rust Frontend] Fix LogprobsTensors wire schema mismatch (#53939)
Signed-off-by: jungjiyu <libraryofjiyu@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-26 16:16:43 -07:00 |
|
|
|
1f9444a34f
|
[Rust Frontend][gRPC] Add LoRA lifecycle control (#52840)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-25 04:13:19 +00:00 |
|
|
Biswa Panda
|
9dba2c9b37
|
[Rust Frontend][RL]: report engine world size over gRPC (#53204)
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
|
2026-08-23 02:31:23 +00:00 |
|
|
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
|
Biswa Pandaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
6a391a931a
|
[Rust Frontend][RL] add routed expert prompt offset (#52703)
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-18 20:04:28 +00:00 |
|
|
Connor CarpenterandBugen Zhao
|
d75136c030
|
[Rust Frontend] Wait for all utility calls to finish (#52671)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-08-18 11:23:34 -07:00 |
|
|
Bugen ZhaoandNick Hill
|
41f179b57a
|
[Rust Frontend] Simplify data-parallel size ownership (#52575)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-18 09:06:05 +00:00 |
|
|
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f08a95f8d8
|
[Rust Frontend][gRPC] Advertise LoRA capabilities (#52031)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-17 13:27:56 -07:00 |
|
|
Connor CarpenterandNick Hill
|
f473870ecf
|
[Rust Frontend][gRPC] Add RL lifecycle control (#51316)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-14 11:32:39 -07:00 |
|
|
|
50ba4bc6b2
|
[Feature] Mask Replay (#49577)
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 08:44:31 +00:00 |
|
|
|
3962042304
|
[Bugfix][V1][Multimodal] Recover from P0/P1 processor cache drift (#46747) (#46747)
Signed-off-by: William Zhao <williamzh@meta.com>
Signed-off-by: WillZZZy <93454470+WillZZZy@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-08-12 12:06:57 +08:00 |
|
|
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
fa722b9f01
|
[Rust Frontend][gRPC] Add explicit data-parallel rank routing (#51178)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-10 15:05:24 -07:00 |
|
|
Karen Chung
|
f57123aa2d
|
feat(frontend): session id plumbing into requests (#48048)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-08-03 22:18:58 +00:00 |
|
|
Connor CarpenterandNick Hill
|
48fc2e2797
|
feat(grpc): add KV event source discovery (#50033)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-29 13:46:40 -07:00 |
|
|
ReidandBugen Zhao
|
242c591d5a
|
[Rust Frontend] Send multimodal tensors in auxiliary frames (#49341)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-29 12:21:09 +00:00 |
|
|
Connor CarpenterandNick Hill
|
e3c2fc3b3c
|
[Rust Frontend][gRPC] Add server and model discovery (#49491)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-27 09:53:27 -07:00 |
|
|
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
59a6b0411d
|
[Core] Fix internal LB load-balancing (#49204)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-27 14:00:46 +00:00 |
|
|
Salt Satoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f25953cc59
|
[Bugfix][Rust Frontend] Handle zero-column logprobs payloads without panicking (#49113)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
|
2026-07-21 04:30:29 +00:00 |
|
|
Connor Carpenterandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
1940c8441e
|
[Rust Frontend][gRPC] Add engine-aware health reporting (#48992)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
|
2026-07-21 10:54:39 +08:00 |
|
|
Sageandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
f1f1259692
|
[Rust Frontend] Use zero-copy slicing for multimodal tensors (#48781)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
|
2026-07-20 16:28:25 +08:00 |
|
|
yzong-rh
|
a287eb163f
|
[Front-end] [Messages] Populate num_cache_creation_tokens (#48535)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-07-18 13:04:35 -04:00 |
|
|
ReidandBugen Zhao
|
f44f3d6f79
|
[Rust Frontend] Wait for mock engine endpoints before ZMQ connect (#47965)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: reidliu41 <reid201711@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-16 09:20:31 +00:00 |
|
|
Bugen Zhao
|
487dfb3418
|
[CI] Add SPDX license header to Rust/Protobuf sources (#48472)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-13 10:22:47 +01:00 |
|
|
omerpaz95andOr Ozeri
|
8df14cfc8c
|
[EC Connector] Add EC Transfer Params (#42433)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-12 14:35:33 +03:00 |
|
|
Reid
|
39a1d32b59
|
[Rust Frontend] Avoid extra copies for multimodal tensors (#47581)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-07-07 03:09:55 +00:00 |
|
|
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
8f4c69b222
|
[Rust Frontend] Cache metric handles for scheduler & request stats (#47444)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-06 13:02:59 +00:00 |
|
|
yangyang-cs95andBugen Zhao
|
6429d5f527
|
[Rust Frontend] add repetition_detection support to sampling params (#46684)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-03 14:06:23 +08:00 |
|
|
Bugen Zhao
|
ec0ffaacc8
|
[Rust Frontend] Improve scheduler stats logging parity (#47435)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 15:55:25 +01:00 |
|
|
Bugen Zhao
|
25fcb65d51
|
[Rust Frontend] Use enum-backed domain types for engine outputs and structured outputs (#47283)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 10:41:46 +01:00 |
|
|
Pranav ThakurandBugen Zhao
|
08a8a4af3f
|
feat(rust): expose profiler control routes in Rust frontend (#46306)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-02 08:46:07 +00:00 |
|
|
Bugen Zhao
|
c638f9216a
|
[Rust Frontend] Split engine core DTOs into separate modules (#47265)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-01 15:28:21 +01:00 |
|
|
Blas Rodriguez Irizar
|
e45c8a9f4b
|
[Rust Frontend] Start current wave for a stale DP FirstRequest (#46833)
Signed-off-by: Blas Rodriguez Irizar <rodrigblas@gmail.com>
|
2026-06-30 05:13:09 +00:00 |
|
|
Reid
|
a16dbd5b85
|
[Rust Frontend] Avoid LoRA registry scans without active LoRA requests (#47040)
Signed-off-by: reidliu41 <reid201711@gmail.com>
|
2026-06-30 04:58:19 +00:00 |
|
|
|
b8cb75b149
|
[Rust Frontend] Add static HTTPS and mTLS support for HTTP and gRPC (#45890)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-30 01:45:59 +00:00 |
|
|
Bugen Zhaoandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ce9f64020b
|
[Rust Frontend] Pass effective reasoning_parser_kwargs for structured output (#46360)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-06-24 03:13:44 +00:00 |
|