[Bugfix][ROCm] Add record_logical_topk_ready to ROCMAiterMLASparseImpl (GLM-5.3-Flash boot crash) (#57252)
Signed-off-by: Mustafa YILDIRIM <mustafa@character.ai> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
co-authored by
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Andreas Karatzas
parent
80447d2765
commit
e0050f287a
@@ -808,6 +808,11 @@ class ROCMAiterMLASparseImpl(
|
||||
(q_concat_shape, vllm_config.model_config.dtype),
|
||||
)
|
||||
|
||||
def record_logical_topk_ready(self) -> None:
|
||||
# This impl shares the top-k indices buffer via SharedTopkIndicesBuffer
|
||||
# but does not participate in sparse-MLA index groups.
|
||||
pass
|
||||
|
||||
def _forward_mla(
|
||||
self,
|
||||
layer: AttentionLayer,
|
||||
|
||||
Reference in New Issue
Block a user