[MoE] Refine FlashInfer one-sided All2All integration (#51924)

Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Bo Li
2026-08-18 00:18:14 +00:00
committed by GitHub
co-authored by mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 0db502c8d8
commit c296851a7d
8 changed files with 129 additions and 68 deletions
@@ -734,17 +734,13 @@ class FlashInferNVLinkOneSidedManager(All2AllManagerBase):
top_k: int,
num_experts: int,
hidden_size: int,
dispatch_dtype_bytes_per_elem: int = 0,
dispatch_scale_bytes_per_token: int = 0,
x_bytes_per_token: int,
x_sf_bytes_per_token: int,
):
"""Initialize (or grow) the MoeAlltoAll workspace."""
if dispatch_dtype_bytes_per_elem == 0:
hidden_bytes = hidden_size // 2
else:
hidden_bytes = hidden_size * dispatch_dtype_bytes_per_elem
total_dispatch_payload_size_per_token = (
hidden_bytes
+ dispatch_scale_bytes_per_token
x_bytes_per_token
+ x_sf_bytes_per_token
+ top_k * 4 # int32 topks ids
+ top_k * 4 # float32 topk weights
)