17 Commits
Author SHA1 Message Date
024c5188bf [Frontend] Resolve logprobs on the parsed streaming derender path (#60437)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
2026-10-16 18:04:15 +08:00
b7b20b1b8f [Bugfix][Frontend] Leave the final stop token out of chat derender text (#61757)
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
2026-10-15 21:51:34 +08:00
Flora FengandChauncey 80dd705c1d [Frontend] Make finish_reason and Responses item status truthful (#59324)
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
2026-10-15 10:12:02 +01:00
8d7b93fec2 [Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251)
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>
Signed-off-by: Shimi Bandiel <shimib@google.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com>
Co-authored-by: Mimir <mimir@nousresearch.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
2026-10-12 19:37:53 +08:00
Ting SUN f4114e068e [Bugfix][Frontend] Preserve token IDs in chat derender responses (#61065)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
2026-10-12 07:55:06 +00:00
7b665f7a58 [Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-10-06 17:22:47 +00:00
Martin Hickey bc21cba967 [Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-10-03 19:38:25 +08:00
Martin Hickey fcfacc3499 [Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-10-02 03:17:56 +00:00
9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-10-02 02:33:00 +00:00
Shimi BandielandClaude 4dc57d44c4 [Frontend] Attach resolved logprobs to streaming derender chunks (#55029)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-29 18:43:44 +00:00
SHREY GAJJAR 5747d4500a [Perf][Frontend] Offload streaming derender detokenization (#57528)
Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com>
2026-09-24 10:39:21 +00:00
Martin Hickey 8b98b7d0b4 [Frontend] Add streaming parity tests and docs for derender (#57922)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-09-21 14:13:52 +00:00
10e6a7f210 [Frontend] Expose per-request spec decode metrics in generate API (#43310)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
2026-09-20 12:21:33 +00:00
Martin Hickey 70164bdadc [Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-09-18 16:36:27 +02:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
2026-08-31 10:13:50 +00:00
Martin Hickey 0a31372e5f [Frontend] Add detokenization streaming derender for disaggregated serving (#47301)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2026-07-29 18:01:20 -07:00