Integrate flashinfer b12x MoE and FP4 GEMM kernels for SM120/121 (#40082)

Signed-off-by: Meenakshi Venkataraman <meenakshiv@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
meena-at-work
2026-05-20 17:21:11 +00:00
committed by GitHub
co-authored by Claude Sonnet 4.6
parent f2d5e3d3ae
commit 363fc84407
10 changed files with 595 additions and 4 deletions
@@ -317,6 +317,7 @@ def prepare_nvfp4_moe_layer_for_fi_or_cutlass(
NvFp4MoeBackend.FLASHINFER_CUTLASS,
NvFp4MoeBackend.FLASHINFER_TRTLLM,
NvFp4MoeBackend.FLASHINFER_CUTEDSL_BATCHED,
NvFp4MoeBackend.FLASHINFER_B12X,
]
# Reorder [w1, w3] to [w3, w1] for FI NVFP4 MoE kernels.
@@ -328,6 +329,7 @@ def prepare_nvfp4_moe_layer_for_fi_or_cutlass(
in [
NvFp4MoeBackend.FLASHINFER_CUTLASS,
NvFp4MoeBackend.FLASHINFER_TRTLLM,
NvFp4MoeBackend.FLASHINFER_B12X,
]
):
w13, w13_scale = reorder_w1w3_to_w3w1(w13, w13_scale)