[ROCm] [Bugfix] Preserve CPU query offsets during capture (#51585)
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
This commit is contained in:
@@ -103,11 +103,9 @@ class RocmAttentionMetadataBuilder(AttentionMetadataBuilder[RocmAttentionMetadat
|
|||||||
# slow, so here we set it to 1.
|
# slow, so here we set it to 1.
|
||||||
attn_metadata.seq_lens.fill_(1)
|
attn_metadata.seq_lens.fill_(1)
|
||||||
|
|
||||||
# Here we set the query start locs to 0. This is to
|
# Zero device query start locations to avoid invalid memory access in
|
||||||
# cover up an invalid memory access in the prefix_prefil kernel
|
# the prefix prefill kernel during graph capture (#25985).
|
||||||
# that we run into during graph capture (#25985)
|
|
||||||
common_attn_metadata.query_start_loc.zero_()
|
common_attn_metadata.query_start_loc.zero_()
|
||||||
common_attn_metadata.query_start_loc_cpu.zero_()
|
|
||||||
|
|
||||||
return attn_metadata
|
return attn_metadata
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user