[Perf] Optimize batch invariant with fused rms norm, 2.1% E2E latency improvement (#40413)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
This commit is contained in:
Wentao Ye
2026-04-21 19:51:03 +00:00
committed by GitHub
parent 6fbec8ed47
commit 16688b26a6
3 changed files with 89 additions and 5 deletions
-4
View File
@@ -61,10 +61,6 @@ def fused_add_rms_norm(
) -> tuple[torch.Tensor, torch.Tensor]:
from vllm import _custom_ops as ops
if envs.VLLM_BATCH_INVARIANT:
return rms_norm_batch_invariant(
x + residual, weight, variance_epsilon
), x + residual
ops.fused_add_rms_norm(
x,
residual,