[ROCm][Perf][DSv4.1] Mono decode on every backbone layer (#61082)

Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Fangzhou Ai
2026-10-13 10:16:50 +08:00
committed by GitHub
co-authored by Claude Opus 5.5
parent f47792af7f
commit cc2a6eea1a
13 changed files with 379 additions and 111 deletions
+6
View File
@@ -291,6 +291,12 @@ class KernelConfig:
enable_jit_warmup: bool = True
"""If True, run JIT compile warmup during kernel warmup."""
monokernel: bool = False
"""If True, run the eligible layers of decode steps on the model's
MonoKernel: a persistent kernel that runs most or all of one decoder layer
in a single launch (DeepSeek-V4.1 on MI355X). A deployment that cannot run
it is refused at startup."""
moe_backend: MoEBackend = "auto"
"""Backend for MoE expert computation kernels. Available options: