[Model] Add PW CUDA graph support for Inkling [2/N] (#48822)

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Woosuk Kwon
2026-07-16 10:28:56 -07:00
committed by GitHub
co-authored by Bugen Zhao Giancarlo Delfin Isotr0py Isotr0py Jee Jee Li Roger Wang Yifan Qiao Claude Fable 5 OpenAI Codex
parent ce65385618
commit 251f7e478e
10 changed files with 98 additions and 32 deletions
@@ -56,7 +56,10 @@ class SpeculatorCudaGraphManager(CudaGraphManager):
block_tables,
attn_groups,
kv_cache_config,
skip_attn=(desc.cg_mode == CUDAGraphMode.PIECEWISE),
skip_attn=(
desc.cg_mode == CUDAGraphMode.PIECEWISE
and not self.use_breakable_cg
),
)
return lambda cg_mode: forward_fn(