[Attention][MLA] Per-request scheduling for MLA chunked context (#50613)

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Matthew Bonanni
2026-08-06 16:45:44 +00:00
committed by GitHub
co-authored by Claude Opus 5 OpenAI Codex mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent e7b8d59460
commit b38e111d3e
15 changed files with 948 additions and 560 deletions
-1
View File
@@ -763,7 +763,6 @@ class MultiHeadLatentAttention(nn.Module, AttentionLayerBase):
prefix_lse=context_lse,
suffix_output=suffix_output[..., : self.v_head_dim],
suffix_lse=suffix_lse,
prefill_tokens_with_context=prefill.chunked_context.prefill_tokens_with_context,
)
elif not writes_out:
out.copy_(output_prefill[..., : self.v_head_dim].flatten(start_dim=-2))