[MoE][Humming] Support SiTU activation for Kimi-K3 (#50510)
Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
This commit is contained in:
co-authored by
OpenAI Codex
Tyler Michael Smith
parent
edbc4969a7
commit
05b7876f50
@@ -251,6 +251,7 @@ class HummingExpertsBase(mk.FusedMoEExpertsModular):
|
||||
MoEActivation.GELU,
|
||||
MoEActivation.GELU_TANH,
|
||||
MoEActivation.SWIGLUOAI,
|
||||
MoEActivation.SITU,
|
||||
MoEActivation.SWIGLUSTEP,
|
||||
MoEActivation.SILU_NO_MUL,
|
||||
MoEActivation.GELU_NO_MUL,
|
||||
|
||||
Reference in New Issue
Block a user