add pad-aware reduce path (#48385)

Signed-off-by: gnovack <novackgm@gmail.com>
This commit is contained in:
gnovack
2026-07-14 18:05:50 -07:00
committed by GitHub
parent 442c421e79
commit f7aadae5e5
7 changed files with 239 additions and 34 deletions
+7 -2
View File
@@ -23,8 +23,13 @@ STABLE_TORCH_LIBRARY_FRAGMENT(_moe_C, m) {
"bias, Tensor? input_ids, Tensor? tid2eid) -> ()");
// Calculate the result of moe by summing up the partial results
// from all selected experts.
m.def("moe_sum(Tensor input, Tensor! output) -> ()");
// from all selected experts. topk_ids/expert_map are optional and, when
// both given, enable pad-aware reduce that skips (token, expert)
// slots that were never actually computed (unrouted, or routed to an
// expert not owned by this rank under expert parallelism).
m.def(
"moe_sum(Tensor input, Tensor! output, Tensor? topk_ids=None, "
"Tensor? expert_map=None) -> ()");
// Aligning the number of tokens to be processed by each expert such
// that it is divisible by the block size.