[Transformers backend] Find attention with a fuser and attach vLLM's layer to it (#54941)
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
co-authored by
Harry Mellor
parent
99a1ab8e4c
commit
8ad2076c4a
@@ -158,4 +158,4 @@ class GLU(nn.Module):
|
||||
|
||||
def test_non_mla_is_not_matched():
|
||||
with torch.device("meta"):
|
||||
assert not isinstance(get_fuser(GLU()), MLAFuser)
|
||||
assert get_fuser(GLU(), MLAFuser) is None
|
||||
|
||||
Reference in New Issue
Block a user