 Martin Hickey
|
7039c7f17b
|
[Bugfix][Rust Frontend] Omit unset sampling params in render output (#61220)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-12 19:31:03 +00:00 |
|
|
Martin Hickey
|
08582942d8
|
[Bugfix][Frontend] Honor request_id and X-Request-Id in chat and completions render (#61236)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-12 19:17:12 +00:00 |
|
|
bzsuni
|
f022975964
|
[Bugfix] Reject beam search in completion render (#61212)
Signed-off-by: bzsuni <bingzhe.sun@daocloud.io>
|
2026-10-12 13:49:46 +00:00 |
|
|
|
8d7b93fec2
|
[Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251)
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com>
Signed-off-by: Shimi Bandiel <shimib@google.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Hyeonki Hong <hhk7734@gmail.com>
Co-authored-by: Mimir <mimir@nousresearch.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-12 19:37:53 +08:00 |
|
|
Ting SUN
|
f4114e068e
|
[Bugfix][Frontend] Preserve token IDs in chat derender responses (#61065)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-10-12 07:55:06 +00:00 |
|
|
aoshen02andOpenAI Codex
|
1789f09588
|
[Frontend] Explain disabled scale-out endpoints with opt-in hints (#61042)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-10-11 08:58:07 +00:00 |
|
|
Harry MellorandClaude Opus 5.5
|
2c99ee9333
|
[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-09 09:29:03 +00:00 |
|
|
Maroon AyoubandClaude
|
8a03ccf7b3
|
[Frontend] Add return_mm_kwargs to render requests (#60728)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-09 08:14:13 +00:00 |
|
|
|
42ecc8eebe
|
[CI][RL] Consolidate RL entrypoint tests under tests/entrypoints/rl (#59948)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-07 08:29:09 +00:00 |
|
|
|
7b665f7a58
|
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-06 17:22:47 +00:00 |
|
|
|
09e0ce3d82
|
[Bugfix][Frontend] Apply model-default reasoning parser in GPU-less render server (#54835)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-06 18:46:24 +08:00 |
|
|
Yida WengandFlora Feng
|
18c8a65edd
|
[BugFix][Frontend] Pass reasoning_ended through render → generate (#60059) (#60062)
Signed-off-by: Yida Weng <84164537+YidaWeng@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-05 20:39:25 +00:00 |
|
|
Ting SUNandFlora Feng
|
50b404e71e
|
[Bugfix][Frontend] Preserve abort finish_reason for scale-out token streams (#47933)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-04 02:57:26 +00:00 |
|
|
|
e319f86f15
|
[Feature] Per-row candidate IDs for prefill token scoring (M2 of #56860) (#56984)
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-03 14:17:09 +00:00 |
|
|
Martin Hickey
|
bc21cba967
|
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-03 19:38:25 +08:00 |
|
|
Martin Hickey
|
fcfacc3499
|
[Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-02 03:17:56 +00:00 |
|
|
|
9bd993564f
|
[Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-10-02 02:33:00 +00:00 |
|
|
|
4a9d6f9572
|
[Bugfix] Preserve sampling masks in DELTA and TITO streaming outputs (#55935)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: fh3 <a@b>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: fh3 <a@b>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-10-01 14:43:38 +08:00 |
|
|
|
3d5625cc3c
|
[Security] Authenticate shared-memory multimodal cache handles (#59357)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-09-30 05:16:26 -07:00 |
|
|
Shimi BandielandClaude
|
4dc57d44c4
|
[Frontend] Attach resolved logprobs to streaming derender chunks (#55029)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-29 18:43:44 +00:00 |
|
|
|
b721a4c709
|
[Feature] Add fixed-token prefill scoring (#54335)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-28 07:50:09 -07:00 |
|
|
Robert ShawandRobert Shaw
|
dfab504333
|
[Bugfix][Frontend] Detect Anthropic inline-system merge against the resolved chat template (#58754)
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
|
2026-09-26 14:00:56 +00:00 |
|
|
SHREY GAJJAR
|
5747d4500a
|
[Perf][Frontend] Offload streaming derender detokenization (#57528)
Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com>
|
2026-09-24 10:39:21 +00:00 |
|
|
aoshen02
|
7f1a5398e9
|
Fix full logprobs in token-in/token-out responses (#58488)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
|
2026-09-24 04:46:16 +00:00 |
|
|
Juan Pérez de Algaba
|
4800024b98
|
[Security] Reject min_tokens that exceeds the filled max_tokens default (#57731)
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
|
2026-09-22 05:50:23 +00:00 |
|
|
Cyrus Leung
|
c8824a1e42
|
[MM] Move supports_multimodal_inputs and cache out of registry (#57913)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-21 07:43:07 -07:00 |
|
|
Martin Hickey
|
8b98b7d0b4
|
[Frontend] Add streaming parity tests and docs for derender (#57922)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-21 14:13:52 +00:00 |
|
|
|
0b7f11a1ee
|
[AuxOutput] Add block-keyed storage for routed-expert outputs (#45635)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-21 10:25:05 +00:00 |
|
|
|
10e6a7f210
|
[Frontend] Expose per-request spec decode metrics in generate API (#43310)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <codex@openai.com>
|
2026-09-20 12:21:33 +00:00 |
|
|
Cyrus Leung
|
56cac80b14
|
[Deprecation] Remove assistant_token_mask support (#57520)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-09-19 00:17:51 +08:00 |
|
|
Martin Hickey
|
70164bdadc
|
[Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-18 16:36:27 +02:00 |
|
|
|
e8a2a0a8bb
|
[Bugfix][Frontend] Reject stop strings on --tokens-only servers instead of silently ignoring them (#57058)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-17 22:02:00 +00:00 |
|
|
aoshen02
|
b978df3299
|
Revert "[Frontend] Omit absent fields from /inference/v1/generate stream chunks" (#57302)
|
2026-09-16 23:31:59 -07:00 |
|
|
aoshen02andClaude Fable 5.1
|
8538017f4b
|
[Frontend] Omit absent fields from /inference/v1/generate stream chunks (#57264)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-17 05:58:56 +00:00 |
|
|
aoshen02
|
0983aef8da
|
[Frontend] Return prompt metadata from /inference/v1/generate (#53187)
|
2026-09-16 15:29:07 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
651a88c09a
|
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 10:57:24 -07:00 |
|
|
Harry MellorandClaude Opus 5
|
8c1557a79c
|
Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:44:29 +00:00 |
|
|
|
caf56ce9da
|
[CI] Pass the scale-out endpoint flag in EC E2E (#56819)
Signed-off-by: Kevin Luu <51931015+khluu@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
|
2026-09-15 12:25:30 +08:00 |
|
|
Martin Hickey
|
934b1fcbcb
|
[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-14 10:02:14 +00:00 |
|
|
shaohuaxiandCodex
|
ae71862c51
|
[Bugfix][Frontend] Check EC requirements for each metadata item (#56070)
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-09-10 19:23:34 +00:00 |
|
|
|
9ffb8cea96
|
[CI] Add e2e test for scale-out EC connector flow (#54973)
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Co-authored-by: zhouyou9505 <zhouyou9505@gmail.com>
Co-authored-by: Cursor Grok 4.6 <noreply@cursor.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
|
2026-09-09 07:38:32 -07:00 |
|
|
Francisco Javier ArceoandOpenAI Codex
|
6a2a2bb02b
|
[Frontend] Add stateless /v1/responses/render endpoint (#50195)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-07 12:50:42 +00:00 |
|
|
zhouyouandCursor Grok 4.6
|
52358e6e19
|
[Frontend] Expose multimodal metadata for disaggregated prefill (#54659)
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com>
Co-authored-by: Cursor Grok 4.6 <noreply@cursor.com>
|
2026-09-06 14:46:52 +00:00 |
|
|
Clinton ThomasandLucas Bourtoule
|
1970f3ed4b
|
Validate scale-out multimodal data before engine handoff (#51898)
Signed-off-by: Clinton Thomas <1033162+KernelClint@users.noreply.github.com>
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
|
2026-09-06 04:58:04 +00:00 |
|
|
|
560ef78bfe
|
[Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU (#54901)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: vx120 <893600387@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: vx120 <893600387@qq.com>
|
2026-09-03 19:03:36 -07:00 |
|
|
Francisco Javier Arceo
|
c00091e026
|
[Frontend] Gate scale-out endpoints behind opt-in flag (#54579)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
|
2026-09-02 14:31:14 +08:00 |
|
|
Hotragn Pettugani
|
ff0c3cb03c
|
[Bugfix][Frontend] Truncate the assistant tokens mask with the prompt (#54539)
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
|
2026-09-01 05:53:22 +00:00 |
|
|
wang.yuqiandCyrus Leung
|
eeb549a74d
|
[Frontend] Move engine/protocol.py out openai folder (#54492)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-08-31 10:13:50 +00:00 |
|
|
Hotragn Pettugani
|
8fd9eb85d5
|
[Bugfix][Frontend] Truncate prompt_token_offsets with the prompt (#54407)
Signed-off-by: hotragn <hotragn.pettugani_2024@woxsen.edu.in>
|
2026-08-31 06:13:37 +00:00 |
|
|
Hyunkyun Moon
|
f25c2692fe
|
[Frontend] Add /v1/messages/render endpoint for the Anthropic Messages API (#45803)
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
|
2026-08-26 11:33:59 +08:00 |
|