[Perf] Support internal prefill checkpoints for Mamba prefix caching, 9%~25% TTFT improvement (#52789)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Ziming Huang <zelda.huanghuang@gmail.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
Wentao Ye
2026-08-22 10:32:40 -07:00
committed by GitHub
co-authored by Ziming Huang OpenAI Codex
parent 020829f66b
commit 9eb9d9d395
12 changed files with 562 additions and 48 deletions
+2 -1
View File
@@ -10,7 +10,8 @@ STABLE_TORCH_LIBRARY(_flashkda_C, m) {
"Tensor(a!) out, Tensor(c!) workspace, Tensor A_log, Tensor dt_bias, "
"float lower_bound, "
"Tensor? initial_state=None, Tensor(b!)? final_state=None, "
"Tensor? cu_seqlens=None) -> ()");
"Tensor? cu_seqlens=None, Tensor(d!)? checkpoint_state=None, "
"Tensor? checkpoint_offsets=None) -> ()");
}
STABLE_TORCH_LIBRARY_IMPL(_flashkda_C, CompositeExplicitAutograd, m) {