8a26869bca
[Bugfix] Index Mamba align state copies by request slot in MRV2 ( #60210 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Karl0007 <44745209+Karl0007@users.noreply.github.com >
Co-authored-by: Yongqin Wang <yongqinwang@roblox.com >
2026-10-07 02:32:46 +00:00
eb2901b735
[Perf][DSv4.1] Run decoder replay layers in CUDA graphs and trim in PIECEWISE graphs ( #59532 )
...
Signed-off-by: Juntian Liu <Juntianl777@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-05 22:11:45 -07:00
Nick Hill
afec265bca
[MRV2] Miscellaneous code cleanup ( #57980 )
2026-09-23 01:51:52 -07:00
hcl and mgoin
71a2392845
[Bugfix][V1] Read ModelState max_model_len from model config ( #58149 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-22 19:45:04 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
3f41d102c5
[1/2][Model Runner V2] DBO support, eager mode ( #50945 )
...
Signed-off-by: specture724 <specture724@gmail.com >
Signed-off-by: Anjie Hou <149605198+specture724@users.noreply.github.com >
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com >
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com >
2026-09-04 14:17:39 -07:00
Canlin Guo and Nick Hill
455edc022b
[ModelRunnerV2] Support prompt embeds ( #42963 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Canlin Guo <961750412@qq.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-17 21:58:50 +00:00
Isotr0py and OpenAI Codex
66728feb1f
[MRV2][Multimodal] Enable encoder cuda graph for model runner v2 ( #49852 )
...
Signed-off-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-08-14 09:33:21 +00:00
Guan-Ming Chiu and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
457a5f34db
[Bugfix][MRV2] Support encoder timing stats in model runner V2 ( #50020 )
...
Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-12 00:00:49 +08:00
Tianyu Guo
dd856e48bb
[Bugfix][EPD][Model Runner V2] Skip gather mm embeddings for encoder only instance ( #51222 )
...
Signed-off-by: Tianyu Guo <guoty@inferact.ai >
2026-08-06 20:40:47 -07:00
Nick Hill and Taneem Ibrahim
fdaa0d9e59
[ModelRunner V2] Support encoder-only attention ( #49331 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com >
2026-07-27 00:38:44 +00:00
zhrrr
61ab70ec3b
[Model Runner V2] support mamba hybrid models align prefix cache ( #42406 )
...
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com >
2026-06-29 14:09:16 -07:00
Nick Hill
c6dd32a810
[ModelRunner V2] Support realtime embeddings ( #46762 )
2026-06-26 19:42:27 -07:00
Nick Hill
b94f212e37
[ModelRunner V2] Deduplicate ModelState init logic ( #46776 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-26 16:32:45 -07:00
Nick Hill and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
84c62e1cbd
[Model Runner V2][MM] Support EVS ( #46535 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-24 10:18:56 -04:00
Nick Hill
fbf520cf3a
[MRV2] Generalize use of WhisperModelState ( #46096 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-22 12:40:02 -07:00
eb28452b10
[Model] Add DiffusionGemma Support ( #45163 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Martin Kukla <martin.kukla@cantab.net >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Dipika Sikka <dsikka@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com >
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com >
2026-06-11 22:17:35 -07:00
Nick Hill
e15f20258b
[ModelRunnerV2] Avoid pipeline parallel bubbles ( #42187 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-02 14:02:01 -07:00
zhrrr
7a08b34fbf
[Model Runner V2] support qwen35 / mamba hybrid model ( #35520 )
...
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com >
2026-05-07 09:31:05 -07:00
Santino Ramos
85f671b8e1
[Model Runner V2] Support Streaming Inputs ( #37028 )
...
Signed-off-by: Santino Ramos <elsantinoramos@gmail.com >
2026-03-20 20:42:25 +00:00
Woosuk Kwon
55eed6b7a5
[Model Runner V2] Add WhisperModelState [6/N] ( #35790 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-03-11 14:20:38 -07:00
Woosuk Kwon
f088a831dd
[Model Runner V2] Use unpadded num_tokens for PW CUDA graph attn metadata ( #36626 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-03-10 09:30:56 -07:00
Woosuk Kwon
e3eb146f7a
[Model Runner V2] Add ModelStateInterface [4/N] ( #35621 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
2026-02-28 13:19:45 -08:00