Francisco Javier Arceo
c00091e026
[Frontend] Gate scale-out endpoints behind opt-in flag ( #54579 )
...
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com >
2026-09-02 14:31:14 +08:00
Nicolò Lucchesi
dafbef15a1
[Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management ( #49445 )
...
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-08-31 14:26:49 +00:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
eligotts and OpenAI Codex
4ae172231c
[Frontend] Forward cache salt for content parts ( #54315 )
...
Signed-off-by: Eli Gotts <78387377+eligotts@users.noreply.github.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-30 23:29:53 -07:00
Taneem Ibrahim
fe6db3ed5e
[Bugfix] Validate scale-out transfer params ( #54324 )
...
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-08-30 09:46:12 -04:00
Juan Pérez de Algaba
1dc464d426
[Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion ( #54353 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-08-30 04:28:16 +00:00
Konstantin Dunas and OpenAI Codex
c71f6f8a81
[Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) ( #42644 )
...
Signed-off-by: hallerite <git@hallerite.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-26 08:42:52 -07:00
Hyunkyun Moon
f25c2692fe
[Frontend] Add /v1/messages/render endpoint for the Anthropic Messages API ( #45803 )
...
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com >
2026-08-26 11:33:59 +08:00
Wei-Cheng (Wayne) Chiu
b41dc4ec72
[Bugfix] Return HTTP 500 for non-streaming generate errors ( #49195 )
...
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com >
2026-08-21 09:07:17 -07:00
Quentin Gallouédec
92bdee05cb
[Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 ( #52399 )
...
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co >
2026-08-19 20:48:45 +08:00
7553aac77b
[Frontend] Add routed-experts prompt offset ( #51906 )
...
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-13 08:09:00 -07:00
50ba4bc6b2
[Feature] Mask Replay ( #49577 )
...
Signed-off-by: vx120 <893600387@qq.com >
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com >
Signed-off-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: aoshen02 <aoshen@inferact.ai >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-13 08:44:31 +00:00
Vinay R Damodaran and Claude Opus 5
3ee2df3033
[Frontend] Make model optional on all /derender request classes ( #51463 )
...
Signed-off-by: Vinay Damodaran <vrdn@hey.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-12 04:48:58 +00:00
33584901ba
[Frontend] Add content_parts to /inference/v1/generate for raw multim… ( #51478 )
...
Signed-off-by: Allen Shen <aoshen@inferact.ai >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-11 10:00:12 +08:00
Karen Chung
f57123aa2d
feat(frontend): session id plumbing into requests ( #48048 )
...
Signed-off-by: Karen Chung <karenc@nvidia.com >
2026-08-03 22:18:58 +00:00
Cyrus Leung
b3f97dae24
[Frontend] Require cache_salt to be non-empty via schema ( #50816 )
...
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk >
2026-08-03 06:51:27 +00:00
Julien Denize
1a20d23dab
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls ( #48947 )
2026-07-30 15:38:00 +02:00
Martin Hickey
0a31372e5f
[Frontend] Add detokenization streaming derender for disaggregated serving ( #47301 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-07-29 18:01:20 -07:00
Martin Hickey
4d4e04f452
[Render] Add round trip parity test and docs for derender ( #48617 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
2026-07-17 05:02:46 +00:00
omerpaz95 and Or Ozeri
8df14cfc8c
[EC Connector] Add EC Transfer Params ( #42433 )
...
Signed-off-by: omerpaz95 <omerpaz95@gmail.com >
Co-authored-by: Or Ozeri <oro@il.ibm.com >
2026-07-12 14:35:33 +03:00
Juan Pérez de Algaba and Cursor
b3e85be663
fix: use configured max_logprobs instead of hardcoded 20 in derender validation ( #47834 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-07-07 09:42:47 +00:00
Juan Pérez de Algaba and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
8e61b646e2
fix(security): add resource bounds validation to derender endpoints ( #47260 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-07 14:58:26 +08:00
wang.yuqi
a14f57a3ac
[Frontend] Refine the entrypoint class's inheritance hierarchy. ( #47498 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-07-03 10:50:06 +00:00
Aleksei Ivashov and NickLucche
024b06b0dc
[Bugfix] Expose usage field in GenerateResponse for disaggregated serving ( #42748 )
...
Signed-off-by: AIvashov <ivashov.aleksey@proton.me >
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai >
Co-authored-by: NickLucche <nicolo.lucchesi@mistral.ai >
2026-07-01 10:00:19 +00:00
wang.yuqi
3483240b7e
[Frontend] Consolidate scale out entrypoints ( #44512 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-06-29 03:18:53 -07:00
Francisco Javier Arceo