[Bugfix][Attention] Make the query contiguous for TRTLLM ragged MLA prefill (#61634)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
355553ac22
commit
3e851a7803
@@ -156,7 +156,8 @@ class TrtllmRaggedPrefillBackend(MLAPrefillBackend):
|
||||
return out
|
||||
|
||||
ret = trtllm_ragged_attention_deepseek(
|
||||
query=q,
|
||||
# The kernel reads the query as packed, ignoring its strides
|
||||
query=q.contiguous(),
|
||||
key=k,
|
||||
value=v,
|
||||
workspace_buffer=self._workspace_buffer,
|
||||
@@ -207,7 +208,8 @@ class TrtllmRaggedPrefillBackend(MLAPrefillBackend):
|
||||
)
|
||||
|
||||
attn_out, lse = trtllm_ragged_attention_deepseek(
|
||||
query=q,
|
||||
# The kernel reads the query as packed, ignoring its strides
|
||||
query=q.contiguous(),
|
||||
key=k,
|
||||
value=v,
|
||||
workspace_buffer=self._workspace_buffer,
|
||||
|
||||
Reference in New Issue
Block a user