[Perf] RMSNorm uncontiguous support, 1.2~3.1x kernel performance improvement (#49750)
Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
This commit is contained in:
@@ -60,7 +60,9 @@ def test_rms_norm(
|
||||
x = x[..., :hidden_size]
|
||||
assert x.is_contiguous() != strided_input
|
||||
x *= scale
|
||||
residual = torch.randn_like(x) * scale if add_residual else None
|
||||
residual = x.new_empty_strided(x.size(), x.stride()) if add_residual else None
|
||||
if residual is not None:
|
||||
residual.normal_(std=scale)
|
||||
|
||||
# NOTE(woosuk): The reference implementation should be executed first
|
||||
# because the custom kernel is in-place.
|
||||
|
||||
Reference in New Issue
Block a user