[DSV4] Fuse norm and router for low latency scenario (#41263)

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: jeejeelee <jeejeelee@verda-b300-05.datacrunch.io>
Co-authored-by: jeejeelee <jeejeelee@verda-b300-05.datacrunch.io>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Jee Jee Li
2026-05-14 05:11:02 -07:00
committed by GitHub
co-authored by jeejeelee mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 1ea9401364
commit 0a65d46628
11 changed files with 815 additions and 43 deletions
+6
View File
@@ -135,6 +135,12 @@ TORCH_LIBRARY_EXPAND(TORCH_EXTENSION_NAME, m) {
// DeepSeek V3 optimized router GEMM for SM90+
m.def("dsv3_router_gemm(Tensor! output, Tensor mat_a, Tensor mat_b) -> ()");
// conditionally compiled so impl registration is in source file
// DeepSeek V4 fused RMSNorm + router GEMV for SM90+
m.def(
"dsv4_norm_router_gemm(Tensor! logits, Tensor! normed_x, Tensor x, "
"Tensor norm_weight, Tensor gate_weight, float eps) -> ()");
// conditionally compiled so impl registration is in source file
#endif
}