[1/2][Model Runner V2] DBO support, eager mode (#50945)

Signed-off-by: specture724 <specture724@gmail.com>
Signed-off-by: Anjie Hou <149605198+specture724@users.noreply.github.com>
Co-authored-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
This commit is contained in:
Anjie Hou
2026-09-04 14:17:39 -07:00
committed by GitHub
co-authored by jiangkuaixue123 Lucas Wilkinson
parent 784cac7c42
commit 3f41d102c5
17 changed files with 1346 additions and 111 deletions
@@ -988,7 +988,9 @@ class DiffusionGemmaModelState(ModelState):
attn_groups,
kv_cache_config,
for_capture=False,
ubatch_idx: int = 0,
) -> dict[str, Any]:
assert ubatch_idx == 0, "DBO is not supported"
if cudagraph_mode == CUDAGraphMode.FULL:
num_reqs = input_batch.num_reqs_after_padding
num_tokens = input_batch.num_tokens_after_padding