[Kernel] Add cuda kernel for gpt_oss activation (#22538)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
This commit is contained in:
Jee Jee Li
2025-08-14 17:21:29 -07:00
committed by GitHub
parent 39cd09dc86
commit 81f4b96481
8 changed files with 150 additions and 24 deletions
+1 -1
View File
@@ -159,7 +159,7 @@ class MLPBlock(torch.nn.Module):
prefix=f"{prefix}.experts",
apply_router_weight_on_input=False,
has_bias=True,
activation="swiglu_oai")
activation="swigluoai")
def forward(self, x: torch.Tensor) -> torch.Tensor:
t = self.norm(x)