[ROCm][Perf][DSv4.1] Mono decode on every backbone layer (#61082)
Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
f47792af7f
commit
cc2a6eea1a
@@ -291,6 +291,12 @@ class KernelConfig:
|
||||
enable_jit_warmup: bool = True
|
||||
"""If True, run JIT compile warmup during kernel warmup."""
|
||||
|
||||
monokernel: bool = False
|
||||
"""If True, run the eligible layers of decode steps on the model's
|
||||
MonoKernel: a persistent kernel that runs most or all of one decoder layer
|
||||
in a single launch (DeepSeek-V4.1 on MI355X). A deployment that cannot run
|
||||
it is refused at startup."""
|
||||
|
||||
moe_backend: MoEBackend = "auto"
|
||||
"""Backend for MoE expert computation kernels. Available options:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user