 Martin Hickey
|
0a31372e5f
|
[Frontend] Add detokenization streaming derender for disaggregated serving (#47301)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-07-29 18:01:20 -07:00 |
|
|
Bugen Zhao
|
f5a7cce9b6
|
[Model] Add Kimi K3 support: Python frontend [2/2] (#50093)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
|
2026-07-29 01:06:01 -07:00 |
|
|
Seiji Eicher
|
6370e53f24
|
[Frontend] Reuse prefill token ids on the decode chat path for disaggregated serving (#48145)
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
|
2026-07-29 10:02:57 +02:00 |
|
|
Guan-Ming Chiu
|
8d28b48d01
|
[Perf] Isolate MM preprocessing on its own executor (#49524)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
|
2026-07-26 08:04:17 +00:00 |
|
|
Guan-Ming Chiu
|
2dc5a72e7e
|
[Bugfix][Renderer] Rebuild vision chunk UUIDs in async render path (#49400)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
|
2026-07-22 14:11:24 +00:00 |
|
|
Guan-Ming Chiu
|
9df2f91232
|
[Renderer] Offload derender CPU work to renderer thread pool (#49396)
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>
|
2026-07-22 06:55:03 +00:00 |
|
|
Wentao Yeandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
877dae9c68
|
[Refactor] Remove deepseek dead code (#48780)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-07-17 14:57:13 +00:00 |
|
|
Martin Hickey
|
4d4e04f452
|
[Render] Add round trip parity test and docs for derender (#48617)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
|
2026-07-17 05:02:46 +00:00 |
|
|
|
6570c9800c
|
[Model] Add Inkling model support [1/N] (#48799)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-07-15 23:40:07 -07:00 |
|
|
Juan Pérez de Algaba
|
793cf79c89
|
[Bugfix][Security] Fix concurrent sparse invariant race bypassing CVE remediation (#48583)
Signed-off-by: jperezde <jperezde@redhat.com>
|
2026-07-14 11:08:24 +00:00 |
|
|
Hoang NguyenandClaude Opus 4.8
|
038ec293b1
|
[Bugfix] Return 400 instead of 500 when multimodal data is sent to a text-only model (#48473)
Signed-off-by: Hoang Nguyen Tien <hoang.nguyentien.2601@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-07-14 08:15:43 +00:00 |
|
|
Muhammad FawazandMahad Durrani
|
d35eba302f
|
[Bugfix] Avoid leaking Pydantic repr in tool_choice error message (#47028)
Signed-off-by: muhammadfawaz1 <135441198+muhammadfawaz1@users.noreply.github.com>
Co-authored-by: Mahad Durrani <114791389+mahadrehmann@users.noreply.github.com>
|
2026-07-08 15:00:59 +08:00 |
|
|
Harry Mellor
|
dd94484577
|
Bump Transformers version to 5.10.4 (#41359)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-07-07 05:13:28 -07:00 |
|
|
wang.yuqi
|
a14f57a3ac
|
[Frontend] Refine the entrypoint class's inheritance hierarchy. (#47498)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-07-03 10:50:06 +00:00 |
|
|
Juan Pérez de Algabaandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
c231d1f290
|
fix(security): bound tokenizer work when explicit truncation_side is set (#47007)
Signed-off-by: jperezde <jperezde@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-06-30 23:08:51 +08:00 |
|
|
wang.yuqi
|
3483240b7e
|
[Frontend] Consolidate scale out entrypoints (#44512)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-29 03:18:53 -07:00 |
|
|
RanranandBenjamin Chislett
|
09841ae705
|
[Render][Speculator] Add return_loss_mask to render endpoint for training data generation (#46846)
Signed-off-by: Ranran Haoran Zhang <ranzhang@redhat.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
|
2026-06-28 00:07:33 -07:00 |
|
|
Brandon PelfreyandRoger Wang
|
af16446bf3
|
Vram semaphore infra (#44465)
Signed-off-by: Brandon Pelfrey <bpelfrey@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
|
2026-06-26 17:32:51 -07:00 |
|
|
Hyunkyun Moon
|
950ee4c2e4
|
[API] Add token offsets to render endpoints (/v1/.../render) (#44226)
Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
|
2026-06-26 05:02:52 -07:00 |
|
|
Tiezhen WANG
|
c7645bce04
|
Remove grok model arch from vllm (#46706)
Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>
|
2026-06-25 23:02:10 -07:00 |
|
|
wang.yuqi
|
31ca9504b1
|
[Frontend] Split ServingRender into renderer and entrypoint. (#44285)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-23 11:19:09 +00:00 |
|
|
wang.yuqi
|
c4fd9794e9
|
[Frontend] Remove AsyncMicrobatchTokenizer. (#45759)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-06-16 08:02:11 +00:00 |
|
|
Harry Mellor
|
3cc9fecd58
|
Deprecated 1st generation Qwen and QwenVL models (#45131)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-06-10 14:55:33 +00:00 |
|
|
|
27f1d34a23
|
[Frontend][Responses API] Move developer-to-system conversion into HF renderer (#43590)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: kdcyberdude <kdsingh.cyberdude@gmail.com>
Co-authored-by: Ben Browning <bbrownin@redhat.com>
|
2026-06-03 14:52:24 +00:00 |
|
|
Gagan Dhakrey
|
1e2ce5d11a
|
offload prompt_embeds decode in render_prompts_async to avoid blocking (#43792)
Signed-off-by: Gagan Dhakrey <gagandhakrey@gmail.com>
|
2026-05-30 01:36:34 +00:00 |
|
|
ruizhangandRui Zhang
|
60af5c16ee
|
[Frontend] Add truncation side to OpenAI endpoints (#43260)
Signed-off-by: Rui Zhang <rza21.bc@gmail.com>
Signed-off-by: Rui Zhang <rui.zhang@globalrelay.net>
Co-authored-by: Rui Zhang <rui.zhang@globalrelay.net>
|
2026-05-21 20:32:31 -07:00 |
|
|
Nick Hill
|
0f66623b0d
|
[Frontend] Rework fastokens integration (#43168)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-21 15:36:58 -07:00 |
|
|
Cyrus Leung
|
56434e8651
|
[Bugfix] Fix incorrect chat template format for Qwen3.5 (#42660)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-05-14 20:52:52 -07:00 |
|
|
TJianandCyrus Leung
|
445d747434
|
[Bugifx] Missing Renderer for fastokens mode (#41984)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-05-07 23:45:14 -07:00 |
|
|
yzong-rhandwang.yuqi
|
20dcd984f9
|
[Bugfix] Fix RuntimeError: Already borrowed by adding thread-safe Hugging Face fast-tokenizer wrappers (#41181)
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
|
2026-05-05 14:04:01 +00:00 |
|
|
Luis 🚀andAndrew Sansom
|
14043dfecd
|
feat: Enable prompt_embeds Content Part Support in vLLM Chat Completions API (#40720)
Signed-off-by: Luis Robaina <luis@protopia.ai>
Signed-off-by: Luis Robaina 🚀 <luisfabian1545@gmail.com>
Signed-off-by: LuisRobaina <luis@protopia.ai>
Co-authored-by: Andrew Sansom <qthequartermasterman@gmail.com>
|
2026-05-01 10:05:55 +08:00 |
|
|
Isotr0py
|
1628239eb2
|
[Multimodal][Render] Skip mm processor initialization and warmup for text-only mode (#41246)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-04-29 14:16:19 -07:00 |
|
|
+7
|
4d51588e23
|
[Feat] DeepSeek V4 Rebased (#40860)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: qizixi <zixi@inferact.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Yongye Zhu <yongye@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com>
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com>
Co-authored-by: khluu <khluu000@gmail.com>
Co-authored-by: qizixi <zixi@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
|
2026-04-26 18:31:08 -07:00 |
|
|
Chenguang ZhengandOpenAI Codex
|
07351e0883
|
[Feature] Warm up readonly multimodal processor during renderer startup (#40797)
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
|
2026-04-25 03:57:41 +00:00 |
|
|
Khushali Desai
|
6ff8dea075
|
[Bugfix] avoid warmup if text only expectation in multi_modal run (#40409)
Signed-off-by: khushali9 <khushali.desai9@gmail.com>
|
2026-04-22 00:19:50 +00:00 |
|
|
Zeyu Zhang
|
5a94a19824
|
[Bugfix] Normalize malformed dict prompts that carry token IDs in prompt (#40339)
Signed-off-by: Alchuang22-dev <2584829494@qq.com>
|
2026-04-21 07:44:36 +00:00 |
|
|
San-Nguyen
|
e729cc823d
|
[Fix] Add Spacing when Requesting Output Token > max_model_len (#40324)
Signed-off-by: San-Nguyen <san.nguyen@ibm.com>
|
2026-04-20 00:25:06 -07:00 |
|
|
Sergey Zinchenko
|
5a2d420c17
|
[Bugfix] Use dedicated MM processor cache in /tokenize to prevent sender-cache pollution (#38545)
Signed-off-by: Sergey Zinchenko <sergey.zinchenko.rnd@gmail.com>
|
2026-04-01 21:14:49 -07:00 |
|
|
BviciiandCyrus Leung
|
999dfc1622
|
[Bugfix] Offload blocking tokenizer ops to shared thread pool to unblock event loop (#34789)
Signed-off-by: Bvicii <yizhanhuang2002@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2026-03-26 22:17:00 -07:00 |
|
|
Cyrus Leung
|
2e225f7bd2
|
[Renderer] Consolidate factory methods (#38218)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-26 12:19:22 +00:00 |
|
|
Cyrus Leung
|
ba2f0acc2d
|
[Misc] Reorganize inputs (#35182)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-25 10:22:54 -07:00 |
|
|
Isotr0py
|
c7f98b4d0a
|
[Frontend] Remove librosa from audio dependency (#37058)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2026-03-21 11:36:15 +08:00 |
|
|
Ekagra Ranjan
|
b5ca9c3557
|
[Models] Cohere ASR (#35809)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
|
2026-03-17 21:04:17 +00:00 |
|
|
Walter Beller-Morales
|
061980c36a
|
[Feature][Frontend] add support for Cohere Embed v2 API (#37074)
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
|
2026-03-16 19:55:53 -04:00 |
|
|
Nick Hill
|
458c1a4b2d
|
[Frontend] Reduce chat template warmup logging levels (#37062)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-03-14 13:48:59 -07:00 |
|
|
Cyrus Leung
|
5467d137b3
|
[Frontend] Avoid startup error log for models without chat template (#37040)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-14 09:36:11 -07:00 |
|
|
|
b41aa264f9
|
fix: resolve chat template names before kwargs detection (#36937)
Co-authored-by: giulio-leone <giulio.leone@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
2026-03-14 00:20:16 +00:00 |
|
|
Cyrus Leung
|
196802dfa6
|
[Misc] Clean up renderers (#36770)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2026-03-11 16:39:29 +00:00 |
|
|
tunglinwood
|
42fadebecb
|
[Model] Add support for moonshotai/Kimi-Audio-7B-Instruct (#36127)
Signed-off-by: tunglinwood <tunglinwood@gmail.com>
Signed-off-by: tunglinwood <tomwu.tunglin@gmail.com>
Signed-off-by: tunglinwood <113751333+tunglinwood@users.noreply.github.com>
|
2026-03-10 21:24:48 -07:00 |
|
|
Harry Mellor
|
f83b933b84
|
[CI] Bump mypy version to 1.19.1 (#36104)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
2026-03-10 09:18:28 -07:00 |
|