[Attention][MLA] Per-request scheduling for MLA chunked context (#50613)
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude Opus 5
OpenAI Codex
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
e7b8d59460
commit
b38e111d3e
@@ -763,7 +763,6 @@ class MultiHeadLatentAttention(nn.Module, AttentionLayerBase):
|
||||
prefix_lse=context_lse,
|
||||
suffix_output=suffix_output[..., : self.v_head_dim],
|
||||
suffix_lse=suffix_lse,
|
||||
prefill_tokens_with_context=prefill.chunked_context.prefill_tokens_with_context,
|
||||
)
|
||||
elif not writes_out:
|
||||
out.copy_(output_prefill[..., : self.v_head_dim].flatten(start_dim=-2))
|
||||
|
||||
Reference in New Issue
Block a user