[MoE Backend] add HPC-Ops MoE backend (#45924)
Signed-off-by: chengvjiang <chengvjiang@tencent.com> Co-authored-by: chengvjiang <chengvjiang@tencent.com> Co-authored-by: youkaichao <youkaichao@gmail.com>
This commit is contained in:
co-authored by
chengvjiang
youkaichao
parent
00e045b7c7
commit
ddd3855a28
@@ -134,6 +134,7 @@ MoEBackend = Literal[
|
||||
"triton_unfused",
|
||||
"aiter",
|
||||
"flydsl",
|
||||
"hpc",
|
||||
"emulation",
|
||||
]
|
||||
|
||||
@@ -189,6 +190,7 @@ class KernelConfig:
|
||||
- "triton_unfused": Use Triton unfused MoE kernels
|
||||
- "aiter": Use AMD AITer kernels (ROCm only)
|
||||
- "flydsl": Use AMD FlyDSL kernels (ROCm only)
|
||||
- "hpc": Use HPC kernels (FP8 and Hopper only)
|
||||
- "emulation": use BF16/FP16 GEMM, dequantizing weights and
|
||||
running QDQ on activations.
|
||||
"""
|
||||
|
||||
Reference in New Issue
Block a user