[ROCm][V1][MLA] Clone prefill backend state per metadata builder (#46993)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
Andreas Karatzas
2026-06-30 11:43:54 -07:00
committed by GitHub
parent 953bba488d
commit c8f9c156a5
3 changed files with 37 additions and 1 deletions
@@ -1563,9 +1563,12 @@ class MLACommonMetadataBuilder(AttentionMetadataBuilder[M]):
device=device,
)
# Metadata builders are created per ubatch when DBO is enabled. MLA
# prefill backends keep the prepared metadata on the backend object, so
# each builder needs its own backend instance to avoid cross-ubatch races.
self._prefill_backend = self.compilation_config.static_forward_context[
layer_names[0]
].prefill_backend
].prefill_backend.clone()
supports_spec_decode = self.query_len_support != QueryLenSupport.SINGLE_ONLY
self._init_reorder_batch_threshold(