[Perf] Warm up DeepGEMM MegaMoE kernels before serving (#60468)
Signed-off-by: Simon Veitner <sveitner@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude
Roberto L. Castro
parent
d9a543c500
commit
253e02b95e
@@ -566,6 +566,7 @@ class KimiMoE(nn.Module):
|
||||
)
|
||||
self.latent_moe_use_norm = config.latent_moe_use_norm
|
||||
self.tp_size = get_tensor_model_parallel_world_size()
|
||||
self.use_sequence_parallel = use_sequence_parallel
|
||||
self.routed_scaling_factor = config.routed_scaling_factor
|
||||
self.moe_renormalize = moe_renormalize
|
||||
self.use_grouped_topk = config.use_grouped_topk
|
||||
|
||||
Reference in New Issue
Block a user