[ROCm][BugFix] Don't use AITER rms_norm as an IR OP on gfx90a when AITER always on. (#59290)
Signed-off-by: Randall Smith <Randall.Smith@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
e73fa00981
commit
2622ee21aa
@@ -1354,12 +1354,14 @@ class RocmPlatform(Platform):
|
||||
default = ["native"] if using_inductor else ["vllm_c", "native"]
|
||||
|
||||
# Aiter rms norm perform best when CUDA Graph capture is enabled.
|
||||
# TODO(luka/TJ) remove env vars completely
|
||||
# TODO(Rohan138/dllehr-amd/rasmith) Change this to be an inclusive
|
||||
# check instead of exclusive check in the future.
|
||||
if (
|
||||
cc.cudagraph_mode != CUDAGraphMode.NONE
|
||||
and envs.VLLM_ROCM_USE_AITER
|
||||
and envs.VLLM_ROCM_USE_AITER_RMSNORM
|
||||
and not on_rdna4()
|
||||
and not on_gfx90a() # AITER not built for gfx90a, rmsnorm -> SEGV.
|
||||
):
|
||||
rms_norm = ["aiter"] + default
|
||||
else:
|
||||
|
||||
Reference in New Issue
Block a user