[Kernel] TD operand loads for batched MoE GEMM (moe_mmk) on XPU (#46340)
Signed-off-by: oonyshch <xonyshch@gmail.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
This commit is contained in:
co-authored by
Kunshang Ji
parent
d30b1ecd1b
commit
0111002323
@@ -122,6 +122,7 @@ class IrOpPriorityConfig:
|
||||
MoEBackend = Literal[
|
||||
"auto",
|
||||
"triton",
|
||||
"batched_triton",
|
||||
"deep_gemm",
|
||||
"deep_gemm_mega_moe",
|
||||
"cutlass",
|
||||
@@ -192,6 +193,8 @@ class KernelConfig:
|
||||
|
||||
- "auto": Automatically select the best backend based on model and hardware
|
||||
- "triton": Use Triton-based fused MoE kernels
|
||||
- "batched_triton": Use batched Triton experts (moe_mmk) on the batched
|
||||
activation format ([E_local, max_num_tokens, K])
|
||||
- "deep_gemm": Use DeepGEMM kernels (FP8 block-quantized only)
|
||||
- "deep_gemm_mega_moe": Use DeepGEMM mega MoE kernels
|
||||
- "cutlass": Use vLLM CUTLASS kernels
|
||||
|
||||
Reference in New Issue
Block a user