 JovanandOpenAI Codex
|
5be4266f5c
|
[Bugfix][Frontend] Preserve Cohere and Anthropic streaming errors (#61947)
Signed-off-by: Kevin <jia__kuan@163.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-17 06:10:16 +00:00 |
|
 Shuolei Wangandaoshen02
|
b3a7e2ff6e
|
[RL][Frontend] Return weight-version spans on OpenAI completions and chat (#61347)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-17 02:08:07 +00:00 |
|
 
|
3dcfaeb4e8
|
[Frontend] Apply each endpoint's top_logprobs cut in derender (#59518)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-15 15:04:04 +00:00 |
|
 Flora FengandChauncey
|
80dd705c1d
|
[Frontend] Make finish_reason and Responses item status truthful (#59324)
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-15 10:12:02 +01:00 |
|
 Bugen ZhaoandClaude Opus 5.5
|
8fa5ae844a
|
[Frontend] Let an unset --tool-strict-level fall back to the tool parser's default (#61328)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-15 15:48:38 +08:00 |
|
wang.yuqi
|
c1692c85f1
|
[Misc] mv chat_utils.py to renderers. (#60522)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-10 06:50:34 +00:00 |
|
 dingdangmaoandFlora Feng
|
bab0ac61f8
|
[Bugfix][Frontend] Forward the reasoning wiring to the engine for batch chat completions (#59263)
Signed-off-by: apex-mochen <2756823972@qq.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 19:32:46 +00:00 |
|
 Ting SUNandFlora Feng
|
4a705c7b54
|
[Bugfix][Frontend] Preserve logprobs when top_logprobs is null (#47838)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-05 22:43:31 +00:00 |
|
 Yida WengandFlora Feng
|
18c8a65edd
|
[BugFix][Frontend] Pass reasoning_ended through render → generate (#60059) (#60062)
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-05 20:39:25 +00:00 |
|
 Stefan WangandRobert Shaw
|
b0eb87fe49
|
[Bugfix][Frontend] Return streaming errors before the first token (#40986)
Signed-off-by: 1fanwang <1fannnw@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
|
2026-10-04 16:42:49 +00:00 |
|
  
|
9bd993564f
|
[Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-02 02:33:00 +00:00 |
|
Flora Feng
|
027b6f3a22
|
[Bugfix][Frontend] Use a fresh parser per choice in non-streaming chat completions (#58939)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-09-28 02:01:35 +00:00 |
|
 
|
8b47e8b22c
|
[Perf][Frontend] Defer reasoning usage recounts for non-continuous chat streams (#56067)
Signed-off-by: Cheng Rui <286040359@qq.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-09-25 14:08:23 -07:00 |
|
  
|
f339e0e757
|
[Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunks (#58583)
Signed-off-by: errmakov <ide404@gmail.com>
Co-authored-by: Yanxiao Zhao <39199723+sdpkjc@users.noreply.github.com>
Co-authored-by: Prakhar Agarwal <270064960+agarwalprakhar2511@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-09-25 16:33:19 +00:00 |
|
Ting SUN
|
b33c8a657d
|
[Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-09-23 14:31:10 +08:00 |
|
 
|
dee37d8911
|
[Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-18 04:32:39 +00:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
Nicolò Lucchesi
|
dafbef15a1
|
[Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-31 14:26:49 +00:00 |
|
 wang.yuqiandCyrus Leung
|
eeb549a74d
|
[Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-08-31 10:13:50 +00:00 |
|
andrewbcohere
|
f94dcde5c5
|
Fix Cohere ChatV2 citation and tool handling issues (#52175)
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
|
2026-08-21 15:23:17 -07:00 |
|
 
|
7cfb97e337
|
[Frontend][Core][Spec Decode] Per-request acceptance stats in OpenAI API responses (#48915)
Signed-off-by: Matthew Kotila <7692737+matthewkotila@users.noreply.github.com>
Co-authored-by: Reed Meyerson <31574681+reed-meyerson@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-20 15:42:49 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
ac2ae8798c
|
[Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-15 06:51:14 -07:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
7553aac77b
|
[Frontend] Add routed-experts prompt offset (#51906)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-13 08:09:00 -07:00 |
|
Chauncey
|
22170354d8
|
[Feature] Parse request priority from HTTP header (#51089)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-08-05 21:10:29 -07:00 |
|
Karen Chung
|
f57123aa2d
|
feat(frontend): session id plumbing into requests (#48048)
Signed-off-by: Karen Chung <karenc@nvidia.com>
|
2026-08-03 22:18:58 +00:00 |
|
 andrewbcohereandCyrus Leung
|
9c110fa522
|
[Frontend] Cohere chat v2 api support (#47189)
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-07-31 22:53:54 -07:00 |
|
wang.yuqi
|
f5ffc59b6a
|
[Renderer] Warm up the renderer properly. (#50408)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-07-31 11:46:53 +00:00 |
|
Julien Denize
|
1a20d23dab
|
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947)
|
2026-07-30 15:38:00 +02:00 |
|
yzong-rh
|
a287eb163f
|
[Front-end] [Messages] Populate num_cache_creation_tokens (#48535)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-07-18 13:04:35 -04:00 |
|
 langzhao-netizenandClaude
|
8b8af2caf7
|
[Frontend] Expose logprob_token_ids on Python OpenAI endpoints (#43463)
Signed-off-by: Lang Zhao <lang.zhao@galileo.ai>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-07-13 14:40:21 -07:00 |
|
 omerpaz95andOr Ozeri
|
8df14cfc8c
|
[EC Connector] Add EC Transfer Params (#42433)
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>
|
2026-07-12 14:35:33 +03:00 |
|
 albertoandChauncey
|
5715fde12c
|
[Feature][Parser] Support include_reasoning param for non-Harmony models (#44301)
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-07-10 15:34:02 +08:00 |
|
 
|
86db6c3070
|
[Frontend] add per-request timing metrics field to response body of Chat/Completions APIs (#46768)
Signed-off-by: Nicholas Edelman <nedelman@nvidia.com>
Signed-off-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: GPT-5.5 <noreply@cursor.com>
|
2026-07-06 17:48:29 -07:00 |
|
wang.yuqi
|
a14f57a3ac
|
[Frontend] Refine the entrypoint class's inheritance hierarchy. (#47498)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-07-03 10:50:06 +00:00 |
|
Chauncey
|
84c2f9f0fb
|
[Frontend] Fix Kimi K2 tool call IDs for required tool choice (#46344)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-06-24 19:59:40 +00:00 |
|
wang.yuqi
|
31ca9504b1
|
[Frontend] Split ServingRender into renderer and entrypoint. (#44285)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-23 11:19:09 +00:00 |
|
Flora Feng
|
0d80979644
|
[Chore] Consolidate reasoning/tool parser attributes into unified Parser in chat serving (#45548)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-15 11:16:45 -04:00 |
|
Ting SUN
|
48df95c43e
|
[Feature][Frontend] Report multimodal token counts in usage.prompt_tokens_details (#45458)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-06-15 05:20:58 +00:00 |
|
 Martin HickeyandIsotr0py
|
43f0e024bc
|
[Render] Add /derender endpoints for disaggregated postprocessing (#43606)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-06-13 13:55:33 +08:00 |
|
 
|
42ae5e7ac6
|
[Bugfix] Fix --enable-prompt-tokens-details omitting zero cached tokens (#44383)
Signed-off-by: Sasindharan Sankar <sasindharansankar@email.com>
Co-authored-by: Sasindharan Sankar <sasindharansankar@email.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-06-11 20:37:42 -07:00 |
|
yzong-rh
|
e0871ad225
|
[Refactor] Chat Completions Streaming Harmony Refactor and Bugfixes (#45104)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-06-12 01:09:47 +00:00 |
|
yzong-rh
|
f712fd0d7d
|
[Refactor] Chat Completions Harmony Refactor, non-streaming path. (#45171)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-06-11 21:18:30 +00:00 |
|
Flora Feng
|
6aec99f030
|
[Refactor] Remove dead states from chat completion serving (#45081)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-09 22:20:15 -07:00 |
|
Flora Feng
|
d82ac00923
|
[Refactor][Mistral] Extract parsing logic into MistralParser (#44596)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-10 00:12:23 +00:00 |
|
wang.yuqi
|
d01d0b4646
|
[Frontend] Consolidate online serving utils. (#44479)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-04 06:49:31 +00:00 |
|
Flora Feng
|
68dafcca75
|
[Refactor] Unify reasoning + tool-call parsing behind Parser.parse() (#44267)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-02 15:11:42 +08:00 |
|
Flora Feng
|
9affc17a05
|
[Refactor] Move unstreamed tool-arg flush from serving layer to parser (#44017)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
2026-06-02 10:37:43 +08:00 |
|
yzong-rh
|
46409fd2a1
|
[Fronten] Clean up stop_token_ids override for Harmony (#44009)
Signed-off-by: Yifan Zong <yzong@redhat.com>
|
2026-05-29 13:28:06 -07:00 |
|
 Aaron HaoandIsotr0py
|
2d6b3489b9
|
[R3] Add routed experts to openai entrypoint (#38939)
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-05-20 09:07:59 -07:00 |
|
aoshen02
|
8c79ad6580
|
Revert "[Core] Replace routing replay with device cache and async D2H pipeline" (#39917) (#42434)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
2026-05-13 23:49:01 -07:00 |
|