[Attention][Perf][Kernel] Replace torch.cat with vectorized CUDA kernel MLA query concat - DeepSeek-V3.2 (#34917)
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com> Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
This commit is contained in:
@@ -2672,6 +2672,21 @@ def cp_gather_and_upconvert_fp8_kv_cache(
|
||||
)
|
||||
|
||||
|
||||
def concat_mla_q(
|
||||
ql_nope: torch.Tensor,
|
||||
q_pe: torch.Tensor,
|
||||
q_out: torch.Tensor,
|
||||
) -> None:
|
||||
"""Concatenate query nope and rope for MLA/DSA attention.
|
||||
|
||||
Args:
|
||||
ql_nope: Query nope component [num_tokens, num_heads, nope_dim]
|
||||
q_pe: Query rope component [num_tokens, num_heads, rope_dim]
|
||||
q_out: Output tensor [num_tokens, num_heads, nope_dim + rope_dim]
|
||||
"""
|
||||
torch.ops._C_cache_ops.concat_mla_q(ql_nope, q_pe, q_out)
|
||||
|
||||
|
||||
def indexer_k_quant_and_cache(
|
||||
k: torch.Tensor,
|
||||
kv_cache: torch.Tensor,
|
||||
|
||||
Reference in New Issue
Block a user