[RL Infra][FlashInfer] Enable router replay output from FlashInfer monolithic MoE kernel (#44214)

Signed-off-by: Xuanyu Zhang <xuanyu.zhang@mistral.ai>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
Xuanyu Zhang
2026-07-20 16:45:10 -07:00
committed by GitHub
co-authored by Claude aoshen02 OpenAI Codex
parent 58b2012aa2
commit 97a668152b
11 changed files with 1097 additions and 10 deletions
@@ -190,6 +190,7 @@ def flashinfer_trtllm_mxint4_moe(
topk_group: int | None = None,
e_score_correction_bias: torch.Tensor | None = None,
routing_method_type: int | None = None,
routing_replay_out: torch.Tensor | None = None,
) -> torch.Tensor:
"""
Apply FlashInfer TensorRT-LLM MxInt4 MoE kernel.
@@ -262,6 +263,7 @@ def flashinfer_trtllm_mxint4_moe(
do_finalize=True,
output=None,
tune_max_num_tokens=8192,
routing_replay_out=routing_replay_out,
)
if isinstance(out, (tuple, list)):
out = out[0]