 Cyrus Leung
|
56cac80b14
|
[Deprecation] Remove assistant_token_mask support (#57520)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-19 00:17:51 +08:00 |
|
|
|
dee37d8911
|
[Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 04:32:39 +00:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
Alexander LeeandOpenAI Codex
|
10e8d5614c
|
[Bugfix][Beam Search] Respect skip_special_tokens during decoding (#56211)
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-14 05:49:44 +00:00 |
|
|
wang.yuqi
|
fe4d4109c2
|
[Frontend][last/N] Move all non-OpenAI content out of the OpenAI folder. (#56369)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-09-11 05:33:26 +00:00 |
|
|
|
ea40bb9e90
|
[feature] Watermarked generation and detection (Gumbel-max algorithm) (#54053)
Signed-off-by: Raphael Rialland <raphael.rialland@mistral.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-09 21:00:15 -07:00 |
|
|
|
e962733e08
|
[Security] Validate cache salts before they reach LMCache (#51444)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-05 03:56:06 +00:00 |
|
|
Nicolò Lucchesi
|
dafbef15a1
|
[Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-31 14:26:49 +00:00 |
|
|
wang.yuqiandCyrus Leung
|
eeb549a74d
|
[Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-08-31 10:13:50 +00:00 |
|
|
Juan Pérez de Algaba
|
1dc464d426
|
[Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion (#54353)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-08-30 04:28:16 +00:00 |
|
|
Kaif KohariandCyrus Leung
|
7fbfca2670
|
[Bugfix][Frontend] Only echo the assistant turn in batched chat completions (#52529)
Signed-off-by: Kaif <kaifkohari10@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-08-29 09:44:17 -07:00 |
|
|
Dieter Schwarzmann
|
585bb07c70
|
feat: add SSE keep-alive comments for idle streaming responses (#51034)
Signed-off-by: fuzzifikation <40043768+fuzzifikation@users.noreply.github.com>
|
2026-08-23 18:13:02 -07:00 |
|
|
andrewbcohere
|
f94dcde5c5
|
Fix Cohere ChatV2 citation and tool handling issues (#52175)
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
|
2026-08-21 15:23:17 -07:00 |
|
|
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
|
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ac2ae8798c
|
[Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-15 06:51:14 -07:00 |
|
|
|
7553aac77b
|
[Frontend] Add routed-experts prompt offset (#51906)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-13 08:09:00 -07:00 |
|
|
|
b64a2708b0
|
Bound generation inputs before expensive work (#51447)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-11 22:03:50 +08:00 |
|
|
Tarun Kumar
|
48bada6ea4
|
Fix chat completion 500 on non-object JSON bodies (#51654)
Signed-off-by: Tarun Kumar <takumar@redhat.com>
|
2026-08-11 01:31:19 +00:00 |
|
|
Chauncey
|
22170354d8
|
[Feature] Parse request priority from HTTP header (#51089)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-08-05 21:10:29 -07:00 |
|
|
Karen Chung
|
f57123aa2d
|
feat(frontend): session id plumbing into requests (#48048)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-08-03 22:18:58 +00:00 |
|
|
Cyrus Leung
|
b3f97dae24
|
[Frontend] Require cache_salt to be non-empty via schema (#50816)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-08-03 06:51:27 +00:00 |
|
|
andrewbcohereandCyrus Leung
|
9c110fa522
|
[Frontend] Cohere chat v2 api support (#47189)
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-31 22:53:54 -07:00 |
|
|
yzong-rhandBen Browning
|
e8b358b4c0
|
[Bugfix][Responses] Add tests for Chat Completions Responses API Render Parity (#50334)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
|
2026-07-31 15:45:40 -04:00 |
|
|
wang.yuqi
|
f5ffc59b6a
|
[Renderer] Warm up the renderer properly. (#50408)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-07-31 11:46:53 +00:00 |
|
|
Julien Denize
|
1a20d23dab
|
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947)
|
2026-07-30 15:38:00 +02:00 |
|
|
Bugen Zhao
|
f5a7cce9b6
|
[Model] Add Kimi K3 support: Python frontend [2/2] (#50093)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-29 01:06:01 -07:00 |
|
|
Walter Beller-Moralesandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
8040ef2426
|
[Frontend] expose stream_interval as req sampling param (#49754)
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-27 11:30:37 +08:00 |
|
|
Umut Polat
|
040cbf95cc
|
[Misc] Use VLLMValidationError in chat completion tool and batch validators (#49214)
Signed-off-by: Umut Polat <52835619+umut-polat@users.noreply.github.com>
|
2026-07-21 11:38:20 +00:00 |
|
|
hcl
|
530ee36a0d
|
fix(openai): reject non-numeric logprobs with 400 instead of 500 (#49144)
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com>
|
2026-07-20 10:04:50 +00:00 |
|
|
yzong-rh
|
ac5f38a0f7
|
[Refactor] Extract StructuredOutputsParams creation logic from Request.to_sampling_params (#49003)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-07-19 05:18:00 -07:00 |
|
|
yzong-rh
|
a287eb163f
|
[Front-end] [Messages] Populate num_cache_creation_tokens (#48535)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-07-18 13:04:35 -04:00 |
|
|
langzhao-netizenandClaude
|
8b8af2caf7
|
[Frontend] Expose logprob_token_ids on Python OpenAI endpoints (#43463)
Signed-off-by: Lang Zhao <lang.zhao@galileo.ai>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-13 14:40:21 -07:00 |
|
|
omerpaz95andOr Ozeri
|
8df14cfc8c
|
[EC Connector] Add EC Transfer Params (#42433)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-12 14:35:33 +03:00 |
|
|
albertoandChauncey
|
5715fde12c
|
[Feature][Parser] Support include_reasoning param for non-Harmony models (#44301)
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-07-10 15:34:02 +08:00 |
|
|
|
86db6c3070
|
[Frontend] add per-request timing metrics field to response body of Chat/Completions APIs (#46768)
Signed-off-by: Nicholas Edelman <nedelman@nvidia.com>
Signed-off-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: GPT-5.5 <noreply@cursor.com>
|
2026-07-06 17:48:29 -07:00 |
|
|
Ting SUN
|
d2afe39647
|
[Bugfix][Frontend] Preserve default sampling params in batch chat (#47597)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-07-04 19:06:39 +00:00 |
|
|
Taneem Ibrahim
|
2f21224527
|
[Misc] Update request-extras parity for batch chat completion (#47333)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-07-04 10:19:04 -04:00 |
|
|
wang.yuqi
|
a14f57a3ac
|
[Frontend] Refine the entrypoint class's inheritance hierarchy. (#47498)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-07-03 10:50:06 +00:00 |
|
|
David FengandClaude
|
18f658bb31
|
[Bugfix][Frontend] Fix batch chat endpoint corrupting logprobs when return_token_ids is set (#47384)
Signed-off-by: David Feng <fenghourun@meta.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-03 03:01:34 -07:00 |
|
|
ganesh
|
fca432e60a
|
[Bugfix] Propagate default stop_token_ids to per-request SamplingParams (#35076)
Signed-off-by: sriganesh123 <arjulasriganesh@gmail.com>
|
2026-06-30 12:10:09 +08:00 |
|
|
RanranandBenjamin Chislett
|
09841ae705
|
[Render][Speculator] Add return_loss_mask to render endpoint for training data generation (#46846)
Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-06-28 00:07:33 -07:00 |
|
|
Ben Browning
|
dccb412e2c
|
[Bugfix][Parser] Pass token IDs to parser.parse() in Responses API and batch serving (#46843)
Signed-off-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-26 19:29:52 +00:00 |
|
|
Hyunkyun Moon
|
950ee4c2e4
|
[API] Add token offsets to render endpoints (/v1/.../render) (#44226)
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
|
2026-06-26 05:02:52 -07:00 |
|
|
Chauncey
|
84c2f9f0fb
|
[Frontend] Fix Kimi K2 tool call IDs for required tool choice (#46344)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-06-24 19:59:40 +00:00 |
|
|
wang.yuqi
|
31ca9504b1
|
[Frontend] Split ServingRender into renderer and entrypoint. (#44285)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-23 11:19:09 +00:00 |
|
|
|
6c427dd401
|
[BugFix] Omit empty tool_calls from OpenAI chat responses (#44105)
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-06-23 13:43:53 +08:00 |
|
|
Flora Feng
|
0d80979644
|
[Chore] Consolidate reasoning/tool parser attributes into unified Parser in chat serving (#45548)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-15 11:16:45 -04:00 |
|
|
Ting SUN
|
48df95c43e
|
[Feature][Frontend] Report multimodal token counts in usage.prompt_tokens_details (#45458)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-15 05:20:58 +00:00 |
|
|
Martin HickeyandIsotr0py
|
43f0e024bc
|
[Render] Add /derender endpoints for disaggregated postprocessing (#43606)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-06-13 13:55:33 +08:00 |
|
|
|
2043258dec
|
[Frontend] Support strict mode for tool calling (#45003)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-12 07:51:48 +00:00 |
|