[Bugfix][MoE] Allow TRTLLM FP8 block-scale MoE with SwiGLU clamp (#55069)

Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen524@gmail.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
This commit is contained in:
aoshen02
2026-09-04 08:34:21 -07:00
committed by GitHub
co-authored by aoshen02 aoshen02 Codex
parent a85d0738da
commit f19431e5c8
2 changed files with 84 additions and 6 deletions
@@ -80,18 +80,26 @@ class TrtLlmFp8ExpertsBase:
)
if not supported:
return supported, reason
# FlashInfer accepts gemm1_alpha/beta/clamp_limit only for MXFP8; the
# DeepSeek-FP8 block-scale kernel rejects them and the per-tensor kernel
# has no clamp at all, so a model-requested SwiGLU clamp cannot be
# honored on those paths.
if (
moe_config.swiglu_limit is not None
or moe_config.swiglu_alpha is not None
or moe_config.swiglu_beta is not None
) and (weight_key, activation_key) != (kMxfp8Static, kMxfp8Dynamic):
) and (
(weight_key, activation_key)
not in (
(kMxfp8Static, kMxfp8Dynamic),
(kFp8Static128BlockSym, kFp8Dynamic128Sym),
)
or moe_config.activation
not in (
MoEActivation.SILU,
MoEActivation.SWIGLUOAI_UNINTERLEAVE,
)
):
return False, (
"the TRTLLM FP8 kernels apply the SwiGLU alpha/beta/clamp "
"parameters only for MXFP8 weights"
"parameters only for block-scaled weights with a SwiGLU "
f"activation, but got {weight_key} and {moe_config.activation}"
)
if moe_config.num_experts > 2048:
return False, (