[Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance improvement (#49750)
Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
This commit is contained in:
@@ -169,8 +169,8 @@ class Lfm2Attention(nn.Module):
|
||||
n_tokens, _ = hidden_states.shape
|
||||
qkv, _ = self.qkv_proj(hidden_states)
|
||||
q, k, v = qkv.split([self.q_size, self.kv_size, self.kv_size], dim=-1)
|
||||
q = q.view(n_tokens, self.num_heads, self.head_dim).contiguous()
|
||||
k = k.view(n_tokens, self.num_kv_heads, self.head_dim).contiguous()
|
||||
q = q.view(n_tokens, self.num_heads, self.head_dim)
|
||||
k = k.view(n_tokens, self.num_kv_heads, self.head_dim)
|
||||
q = self.q_layernorm(q)
|
||||
k = self.k_layernorm(k)
|
||||
q, k = self.rotary_emb(positions, q, k)
|
||||
|
||||
Reference in New Issue
Block a user