[NVFP4] NVFP4 MOE emulation fallback for H100/MI300/MI350, standardize TritonExperts usage for OCP MX emulation (#35737)
Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: fxmarty-amd <felmarty@amd.com> Co-authored-by: Kyle Sayers <kylesayrs@gmail.com>
This commit is contained in:
co-authored by
Kyle Sayers
parent
5f76b3fb30
commit
d622e27d2b
@@ -115,6 +115,7 @@ MoEBackend = Literal[
|
||||
"flashinfer_cutedsl",
|
||||
"marlin",
|
||||
"aiter",
|
||||
"emulation",
|
||||
]
|
||||
|
||||
|
||||
@@ -142,7 +143,10 @@ class KernelConfig:
|
||||
- "flashinfer_cutlass": Use FlashInfer with CUTLASS kernels
|
||||
- "flashinfer_cutedsl": Use FlashInfer with CuteDSL kernels (FP4 only)
|
||||
- "marlin": Use Marlin kernels (weight-only quantization)
|
||||
- "aiter": Use AMD AITer kernels (ROCm only)"""
|
||||
- "aiter": Use AMD AITer kernels (ROCm only)
|
||||
- "emulation": use BF16/FP16 GEMM, dequantizing weights and
|
||||
running QDQ on activations.
|
||||
"""
|
||||
|
||||
@field_validator("moe_backend", mode="before")
|
||||
@classmethod
|
||||
|
||||
Reference in New Issue
Block a user