[Quantization] Add per-token NVFP4 CuTe-DSL MoE backend (#50030)
Signed-off-by: aoshen02 <aoshen@inferact.ai> Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
This commit is contained in:
co-authored by
aoshen02
OpenAI Codex
aoshen02
parent
d5ee309314
commit
0103a9b96f
@@ -720,7 +720,7 @@ class MoERunner(MoERunnerInterface):
|
||||
)
|
||||
|
||||
# Record before `_maybe_pad_hidden_states` pads activations to match
|
||||
# `moe_config.hidden_dim`, e.g. after `align_trtllm_fp4_moe_hidden_dim_for_fi`
|
||||
# `moe_config.hidden_dim`, e.g. after `align_fp4_moe_hidden_dim_for_fi`
|
||||
# so routed output can be trimmed before
|
||||
# shared+routed add / latent up proj if needed.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user