[Hardware][CPU] Enable granite-4 model on cpu (#47641)
Signed-off-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com> Signed-off-by: Akash Kaothalkar <akashkaothalkar@dhcp-9-123-5-76.bl1-in.ibm.com> Signed-off-by: Akash Kaothalkar <akashkaothalkar@Akashs-MBP.lan> Signed-off-by: Akash kaothalkar <akash.kaothalkar@ibm.com> Co-authored-by: Akash Kaothalkar <akashkaothalkar@dhcp-9-123-5-76.bl1-in.ibm.com> Co-authored-by: Akash Kaothalkar <akashkaothalkar@Akashs-MBP.lan> Co-authored-by: Akash Kaothalkar <akashkaothalkar@akashs-mbp.bl1-in.ibm.com> Co-authored-by: Akash kaothalkar <akash.kaothalkar@ibm.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com>
This commit is contained in:
co-authored by
Akash Kaothalkar
Akash Kaothalkar
Akash Kaothalkar
Akash kaothalkar
Li, Jiang <jiang1.li@intel.com>
parent
dcfebf93f4
commit
5c9f6557d7
@@ -234,10 +234,14 @@ def dispatch_cpu_unquantized_gemm(
|
||||
layer.cpu_linear = torch.nn.functional.linear
|
||||
return
|
||||
|
||||
# Skip CPU GEMM dispatch for non-2D weights (e.g. MoE 3D expert weights).
|
||||
# These layers are handled by their own specialized methods.
|
||||
if layer.weight.ndim != 2:
|
||||
# this is not a linear layer
|
||||
# For now it should be a causal_conv1d op
|
||||
if torch.cpu._is_amx_tile_supported():
|
||||
# For now it should be a causal_conv1d op or MoE 3D expert weights
|
||||
if torch.cpu._is_amx_tile_supported() and hasattr(
|
||||
ops, "causal_conv1d_weight_pack"
|
||||
):
|
||||
# prepack conv weight
|
||||
unpacked = (
|
||||
layer.weight.view(
|
||||
|
||||
Reference in New Issue
Block a user