[GLM 5.3 Perf] Enable fused multi-step decode, 13.3% E2E throughput improvement for concurrency 1 (#57443)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
This commit is contained in:
Wentao Ye
2026-10-02 20:05:27 -04:00
committed by GitHub
parent 6e517b15c1
commit 8bdd8d847d
7 changed files with 149 additions and 0 deletions
@@ -373,6 +373,7 @@ class DraftModelSpeculator(BaseSpeculator):
kv_cache_config=self.kv_cache_config,
causal=causal,
seq_lens_cpu_upper_bound=draft_seq_lens_cpu_upper_bound,
positions=self.input_buffers.positions[:num_tokens],
is_prefilling=self.draft_is_prefilling[:num_reqs_padded],
)
return attn_metadata