 
|
3dcfaeb4e8
|
[Frontend] Apply each endpoint's top_logprobs cut in derender (#59518)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-15 15:04:04 +00:00 |
|
Martin Hickey
|
252be28c18
|
[Frontend] Replace return_mm_kwargs with an mm_payload enum on render requests (#61427)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-15 12:41:46 +00:00 |
|
Martin Hickey
|
c6b772cf6b
|
[Bugfix][Frontend] Emit kv/ec transfer params at the top level of render output (#61241)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-15 12:34:15 +00:00 |
|
Martin Hickey
|
08582942d8
|
[Bugfix][Frontend] Honor request_id and X-Request-Id in chat and completions render (#61236)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-12 19:17:12 +00:00 |
|
bzsuni
|
f022975964
|
[Bugfix] Reject beam search in completion render (#61212)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
|
2026-10-12 13:49:46 +00:00 |
|
   
|
8d7b93fec2
|
[Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251)
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>
Signed-off-by: Shimi Bandiel <shimib@google.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com>
Co-authored-by: Mimir <mimir@nousresearch.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-12 19:37:53 +08:00 |
|
 aoshen02andOpenAI Codex
|
1789f09588
|
[Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-11 08:58:07 +00:00 |
|
 Shuolei Wangandaoshen02
|
b89852aad0
|
[RL][Frontend] Return token-level weight-version spans on final outputs (#59582)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-11 11:33:29 +08:00 |
|
wang.yuqi
|
c1692c85f1
|
[Misc] mv chat_utils.py to renderers. (#60522)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-10 06:50:34 +00:00 |
|
 Maroon AyoubandClaude
|
8a03ccf7b3
|
[Frontend] Add return_mm_kwargs to render requests (#60728)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-09 08:14:13 +00:00 |
|
 
|
7b665f7a58
|
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-06 17:22:47 +00:00 |
|
 Yida WengandFlora Feng
|
18c8a65edd
|
[BugFix][Frontend] Pass reasoning_ended through render → generate (#60059) (#60062)
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-05 20:39:25 +00:00 |
|
 Ting SUNandFlora Feng
|
50b404e71e
|
[Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams (#47933)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-04 02:57:26 +00:00 |
|
Martin Hickey
|
bc21cba967
|
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-03 19:38:25 +08:00 |
|
Martin Hickey
|
fcfacc3499
|
[Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-02 03:17:56 +00:00 |
|
  
|
4a9d6f9572
|
[Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-10-01 14:43:38 +08:00 |
|
 
|
3d5625cc3c
|
[Security] Authenticate shared-memory multimodal cache handles (#59357)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-30 05:16:26 -07:00 |
|
 Shimi BandielandClaude
|
4dc57d44c4
|
[Frontend] Attach resolved logprobs to streaming derender chunks (#55029)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-29 18:43:44 +00:00 |
|
 
|
b721a4c709
|
[Feature] Add fixed-token prefill scoring (#54335)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-28 07:50:09 -07:00 |
|
 Robert ShawandRobert Shaw
|
dfab504333
|
[Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
|
2026-09-26 14:00:56 +00:00 |
|
aoshen02
|
7f1a5398e9
|
Fix full logprobs in token-in/token-out responses (#58488)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
2026-09-24 04:46:16 +00:00 |
|
Martin Hickey
|
8b98b7d0b4
|
[Frontend] Add streaming parity tests and docs for derender (#57922)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-21 14:13:52 +00:00 |
|
 
|
10e6a7f210
|
[Frontend] Expose per-request spec decode metrics in generate API (#43310)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-20 12:21:33 +00:00 |
|
Cyrus Leung
|
56cac80b14
|
[Deprecation] Remove assistant_token_mask support (#57520)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-19 00:17:51 +08:00 |
|
Martin Hickey
|
70164bdadc
|
[Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-18 16:36:27 +02:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png)  
|
e8a2a0a8bb
|
[Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-17 22:02:00 +00:00 |
|
aoshen02
|
b978df3299
|
Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302)
|
2026-09-16 23:31:59 -07:00 |
|
 aoshen02andClaude Fable 5.1
|
8538017f4b
|
[Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-17 05:58:56 +00:00 |
|
aoshen02
|
0983aef8da
|
[Frontend] Return prompt metadata from /inference/v1/generate (#53187)
|
2026-09-16 15:29:07 -07:00 |
|
 Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
 Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
Martin Hickey
|
934b1fcbcb
|
[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-14 10:02:14 +00:00 |
|
 shaohuaxiandCodex
|
ae71862c51
|
[Bugfix][Frontend] Check EC requirements for each metadata item (#56070)
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-10 19:23:34 +00:00 |
|
 Francisco Javier ArceoandOpenAI Codex
|
6a2a2bb02b
|
[Frontend] Add stateless /v1/responses/render endpoint (#50195)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-07 12:50:42 +00:00 |
|
 zhouyouandCursor Grok 4.6
|
52358e6e19
|
[Frontend] Expose multimodal metadata for disaggregated prefill (#54659)
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com>
Co-authored-by: Cursor Grok 4.6 <noreply@cursor.com>
|
2026-09-06 14:46:52 +00:00 |
|
 Clinton ThomasandLucas Bourtoule
|
1970f3ed4b
|
Validate scale-out multimodal data before engine handoff (#51898)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
|
2026-09-06 04:58:04 +00:00 |
|
 
|
e962733e08
|
[Security] Validate cache salts before they reach LMCache (#51444)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-05 03:56:06 +00:00 |
|
Francisco Javier Arceo
|
c00091e026
|
[Frontend] Gate scale-out endpoints behind opt-in flag (#54579)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
|
2026-09-02 14:31:14 +08:00 |
|
Nicolò Lucchesi
|
dafbef15a1
|
[Core] Add max_num_queued_reqs and max_num_queued_tokens for queue size management (#49445)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-08-31 14:26:49 +00:00 |
|
 wang.yuqiandCyrus Leung
|
eeb549a74d
|
[Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-08-31 10:13:50 +00:00 |
|
 eligottsandOpenAI Codex
|
4ae172231c
|
[Frontend] Forward cache salt for content parts (#54315)
Signed-off-by: Eli Gotts <78387377+eligotts@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-30 23:29:53 -07:00 |
|
Taneem Ibrahim
|
fe6db3ed5e
|
[Bugfix] Validate scale-out transfer params (#54324)
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
|
2026-08-30 09:46:12 -04:00 |
|
Juan Pérez de Algaba
|
1dc464d426
|
[Bugfix] Bound cache_salt length to prevent DoS via scheduler CPU exhaustion (#54353)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-08-30 04:28:16 +00:00 |
|
 Konstantin DunasandOpenAI Codex
|
c71f6f8a81
|
[Bugfix] Thread kv_transfer_params into engine for /inference/v1/generate (disagg) (#42644)
Signed-off-by: hallerite <git@hallerite.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-08-26 08:42:52 -07:00 |
|
Hyunkyun Moon
|
f25c2692fe
|
[Frontend] Add /v1/messages/render endpoint for the Anthropic Messages API (#45803)
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
|
2026-08-26 11:33:59 +08:00 |
|
Wei-Cheng (Wayne) Chiu
|
b41dc4ec72
|
[Bugfix] Return HTTP 500 for non-streaming generate errors (#49195)
Signed-off-by: WEI CHENG CHIU <waynehacking8@gmail.com>
|
2026-08-21 09:07:17 -07:00 |
|
Quentin Gallouédec
|
92bdee05cb
|
[Bugfix][Frontend] Return all choices from /inference/v1/generate when n > 1 (#52399)
Signed-off-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
|
2026-08-19 20:48:45 +08:00 |
|
![mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>](/assets/img/avatar_default.png) 
|
7553aac77b
|
[Frontend] Add routed-experts prompt offset (#51906)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-08-13 08:09:00 -07:00 |
|
   
|
50ba4bc6b2
|
[Feature] Mask Replay (#49577)
Signed-off-by: vx120 <893600387@qq.com>
Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-08-13 08:44:31 +00:00 |
|
 Vinay R DamodaranandClaude Opus 5
|
3ee2df3033
|
[Frontend] Make model optional on all /derender request classes (#51463)
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-08-12 04:48:58 +00:00 |
|