 Woosuk KwonandClaude Opus 5.5
|
e57bb4ace7
|
[Core] Remove the legacy Ray compiled-graph executor (#61712)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-14 18:13:27 -07:00 |
|
|
wang.yuqi
|
6ecb39eaa7
|
[Docs] Update structured decisions docs. (#61588)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-10-14 10:28:22 +00:00 |
|
|
|
c923e7769b
|
[Feat] Add text-only OpenAI Decisions API MVP (#60465)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-14 07:51:08 +08:00 |
|
|
|
380638afbb
|
[Model] Add Clef-Flash support (#61308)
Signed-off-by: Tianyu Guo <guoty@inferact.ai>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-13 05:46:37 +00:00 |
|
|
|
fa39582756
|
[Frontend] Serve LiquidAI/d1-3B on /v1/systemone (#60676)
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-12 13:00:47 -04:00 |
|
|
|
092c35dd15
|
[Model] Support Cloudflare/clef-omni on /v1/systemone (#61064)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
|
2026-10-12 21:27:34 +08:00 |
|
|
|
4163a80142
|
[Rust Frontend] Return weight-version spans on final outputs (#61070)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-10-12 15:56:59 +08:00 |
|
|
Shuolei Wangandaoshen02
|
b89852aad0
|
[RL][Frontend] Return token-level weight-version spans on final outputs (#59582)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
|
2026-10-11 11:33:29 +08:00 |
|
|
|
3c802af3cf
|
[Feat][Frontend] structured decisions: DiffusionGemma read strategy (#60466)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-10 09:33:22 +08:00 |
|
|
Taewoong KimandCyrus Leung
|
75e3b17c05
|
[Frontend] Add cache salt support to generative scoring (#60841)
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-09 16:07:50 +00:00 |
|
|
Taewoong KimandChauncey
|
18c19eaf23
|
[Frontend] Add cache salt and tracing to structured decisions (#60786)
Signed-off-by: Taewoong Kim <ktw2172@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-09 08:53:50 +00:00 |
|
|
Maroon AyoubandClaude
|
8a03ccf7b3
|
[Frontend] Add return_mm_kwargs to render requests (#60728)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-09 08:14:13 +00:00 |
|
|
|
8cbd5d0300
|
[Frontend] structured decisions: noul and score question types (#60696)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Co-authored-by: DeepSeek <noreply@deepseek.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
|
2026-10-09 04:48:16 +00:00 |
|
|
sungbin1015andFlora Feng
|
2ddde19e8e
|
[Frontend] Support min_p in the Responses API (#48084)
Signed-off-by: sungbin1015 <sbin@solbox.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
|
2026-10-08 20:38:17 +00:00 |
|
|
|
ae9adec905
|
[Frontend] Expose /v1/systemone without an opt-in flag (#60677)
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-10-08 20:21:05 +00:00 |
|
|
|
d33872dfeb
|
[Feat] Structured decisions endpoint (/v1/systemone) (#59299)
Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Francisco Javier Arceo <4163062+franciscojavierarceo@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
|
2026-10-08 14:48:12 +00:00 |
|
|
|
7b665f7a58
|
[Frontend] Integer token IDs for generate output logprobs (GenerateLogProbs) (#58181)
Signed-off-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Yin Li <kevinli.ai.work@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-10-06 17:22:47 +00:00 |
|
|
Martin Hickey
|
bc21cba967
|
[Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) (#58588)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-03 19:38:25 +08:00 |
|
|
Julien Debache
|
111f71a6db
|
[feat] FlashInfer CuteDSL MegaMoE integration (#54049)
Signed-off-by: jdebache <jdebache@nvidia.com>
|
2026-10-02 15:43:33 -04:00 |
|
|
Martin Hickey
|
fcfacc3499
|
[Bugfix][Frontend] Seed derender detokenization from the prompt (#59046)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-10-02 03:17:56 +00:00 |
|
|
Sundri Lai
|
bcee730b1a
|
[Docs] Remove references to removed env vars (#59530)
Signed-off-by: Sundri Lai <laiyanting.neu@gmail.com>
|
2026-10-01 08:26:14 +00:00 |
|
|
Shimi BandielandClaude
|
4dc57d44c4
|
[Frontend] Attach resolved logprobs to streaming derender chunks (#55029)
Signed-off-by: Shimi Bandiel <shimib@google.com>
Co-authored-by: Claude <noreply@anthropic.com>
|
2026-09-29 18:43:44 +00:00 |
|
|
|
7a9406690e
|
[KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472)
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-09-24 23:55:47 +00:00 |
|
|
Tony Lin
|
dc479a2d83
|
[Bugfix] Fix external LB DP rank handling when replicas share nodes (#53743)
Signed-off-by: Tony Lin <tony.lin@intel.com>
|
2026-09-21 17:19:14 +02:00 |
|
|
Martin Hickey
|
8b98b7d0b4
|
[Frontend] Add streaming parity tests and docs for derender (#57922)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-21 14:13:52 +00:00 |
|
|
Martin Hickey
|
70164bdadc
|
[Frontend] Add stream reasoning and tool calls from the derender endpoint (#50550)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-18 16:36:27 +02:00 |
|
|
|
f37c550bf6
|
[Feature][Frontend] Expose effective attention block size for DCP (#56538)
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-09-16 08:29:43 +00:00 |
|
|
Francisco Javier ArceoandOpenAI Codex
|
7d4bba5498
|
[Doc] Show how to get Responses prompt token IDs (#55912)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-16 02:48:18 +00:00 |
|
|
Martin Hickey
|
934b1fcbcb
|
[Frontend] Replace VLLM_ENABLE_SCALE_OUT_ENDPOINTS with --enable-scale-out (#55176)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-09-14 10:02:14 +00:00 |
|
|
Nicolò Lucchesi
|
cd64c2dea9
|
[Docs] Clarify admission control limits apply server-wide, not per DP rank (#55124)
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
|
2026-09-07 14:08:48 +00:00 |
|
|
Francisco Javier ArceoandOpenAI Codex
|
6a2a2bb02b
|
[Frontend] Add stateless /v1/responses/render endpoint (#50195)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-09-07 12:50:42 +00:00 |
|
|
zhouyouandCursor Grok 4.6
|
52358e6e19
|
[Frontend] Expose multimodal metadata for disaggregated prefill (#54659)
Signed-off-by: zhouyou9505 <zhouyou9505@gmail.com>
Co-authored-by: Cursor Grok 4.6 <noreply@cursor.com>
|
2026-09-06 14:46:52 +00:00 |
|
|
Francisco Javier Arceo
|
c00091e026
|
[Frontend] Gate scale-out endpoints behind opt-in flag (#54579)
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
|
2026-09-02 14:31:14 +08:00 |
|
|
|
a1c5b1fd9f
|
[Core][V1] Support trace_decode_token_ids for deterministic decode replay (#46701)
Signed-off-by: ianzxliu <ianzxliu@tencent.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Zhixia Liu <liuzhixia.nju@gmail.com>
Co-authored-by: ianzxliu <ianzxliu@tencent.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-20 15:12:43 +08:00 |
|
|
Russell BryantandClaude Opus 4.8
|
8151f2ad43
|
[Docs] Warn that --api-key does not gate all endpoints (#51999)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-08-12 08:16:35 -07:00 |
|
|
Chauncey
|
22170354d8
|
[Feature] Parse request priority from HTTP header (#51089)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
|
2026-08-05 21:10:29 -07:00 |
|
|
christian
|
9d1aa4dda8
|
[Frontend] Add diarized_json support for MOSS-Transcribe-Diarize (#48543)
Signed-off-by: Lucas Christian <lucaschgf7@gmail.com>
|
2026-07-29 23:24:30 +00:00 |
|
|
Shuolei Wang
|
9069a57139
|
[Core][Frontend] Add weight version tagging for RL rollouts (#49040)
Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com>
Signed-off-by: Shuolei Wang <948904026@qq.com>
|
2026-07-28 14:35:29 +08:00 |
|
|
Woosuk Kwon
|
60b3d39cd3
|
[Docs] Remove experimental warning for EP (#50057)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-07-27 17:06:28 -07:00 |
|
|
|
5d8e90a966
|
[WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@vllm.ai>
Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Codex <noreply@openai.com>
|
2026-07-24 13:00:02 -07:00 |
|
|
Euisuh Jeong
|
da54a5bf05
|
[Docs] Fix broken anchor links in serving/pooling/MoE docs (#49654)
Signed-off-by: euisuh <euisuh.jeong@gmail.com>
|
2026-07-24 02:42:56 +00:00 |
|
|
Lena OnyshchenkoandCursor
|
1d874867ea
|
[Misc][Docs] Fix broken protocol link in speech_to_text doc (#47212)
Signed-off-by: oonyshch <xonyshch@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-07-21 01:05:05 +00:00 |
|
|
Martin Hickey
|
4d4e04f452
|
[Render] Add round trip parity test and docs for derender (#48617)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-07-17 05:02:46 +00:00 |
|
|
Daoyuan Li
|
b88abb5036
|
[Misc] Remove orphaned env vars and stale env-var references (#44749)
Signed-off-by: Daoyuan Li <94409450+DaoyuanLi2816@users.noreply.github.com>
|
2026-07-17 00:00:47 +00:00 |
|
|
|
83762b77b0
|
[Frontend] Add /abort_requests to the RLHF dev API router (#47173)
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-12 14:21:02 +08:00 |
|
|
wang.yuqi
|
3483240b7e
|
[Frontend] Consolidate scale out entrypoints (#44512)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-29 03:18:53 -07:00 |
|
|
wang.yuqi
|
1d88c4dadd
|
[Docs] Update the online serving docs. (#45676)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-15 17:23:36 +08:00 |
|
|
wang.yuqi
|
b623f7ea95
|
[Frontend] Consolidate dev entrypoints. (#44170)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-02 06:30:21 -07:00 |
|
|
|
4aaba00f92
|
[EPLB] Make async EPLB default (#43219)
Signed-off-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Markov Ilya <markovilya19@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
|
2026-05-29 18:07:16 +00:00 |
|
|
wang.yuqiandgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
1b26fa361e
|
[Docs] Reorganize offline inference docs. (#43552)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2026-05-25 13:44:39 +08:00 |
|