[DSV4] Fuse norm and router for low latency scenario (#41263)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: jeejeelee <jeejeelee@verda-b300-05.datacrunch.io> Co-authored-by: jeejeelee <jeejeelee@verda-b300-05.datacrunch.io> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
jeejeelee
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
1ea9401364
commit
0a65d46628
@@ -135,6 +135,12 @@ TORCH_LIBRARY_EXPAND(TORCH_EXTENSION_NAME, m) {
|
||||
// DeepSeek V3 optimized router GEMM for SM90+
|
||||
m.def("dsv3_router_gemm(Tensor! output, Tensor mat_a, Tensor mat_b) -> ()");
|
||||
// conditionally compiled so impl registration is in source file
|
||||
|
||||
// DeepSeek V4 fused RMSNorm + router GEMV for SM90+
|
||||
m.def(
|
||||
"dsv4_norm_router_gemm(Tensor! logits, Tensor! normed_x, Tensor x, "
|
||||
"Tensor norm_weight, Tensor gate_weight, float eps) -> ()");
|
||||
// conditionally compiled so impl registration is in source file
|
||||
#endif
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user