[ROCm][Bugfix] Fall back from CK MXFP4 MoE when GEMM dimensions are unsupported (#35893)
Signed-off-by: Li <chuali@amd.com>
This commit is contained in:
@@ -14,6 +14,13 @@ from vllm.utils.torch_utils import direct_register_custom_op, is_torch_equal_or_
|
||||
|
||||
logger = init_logger(__name__)
|
||||
|
||||
# CK's pre-compiled MXFP4 MoE GEMM kernel instances require the
|
||||
# intermediate_size (after TP split) to be a multiple of this value.
|
||||
# This arises from FP4 packing (2 values per byte) combined with CK
|
||||
# tile size constraints. When violated, AITER raises:
|
||||
# "device_gemm ... does not support this GEMM problem".
|
||||
CK_MXFP4_MOE_DIM_ALIGNMENT = 256
|
||||
|
||||
|
||||
def _swizzle_mxfp4(quant_tensor, scale, num_warps):
|
||||
"""weight swizzle for mxfp4 moe, used for OAI mxfp4 kernel"""
|
||||
|
||||
Reference in New Issue
Block a user