[Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250)

Signed-off-by: Matt Mastracci <matthew@mastracci.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Razorback16 <razorback16@protonmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Razorback16 <razorback16@protonmail.com>
This commit is contained in:
Matt Mastracci
2026-09-22 20:51:17 +00:00
committed by GitHub
co-authored by Claude Fable 5.1 Lucas Wilkinson Codex Razorback16
parent fae5cbfc8b
commit 1b3b88ec2b
16 changed files with 2795 additions and 155 deletions
+6 -3
View File
@@ -280,11 +280,14 @@ class DiffusionGemmaModelForBlockDiffusionConfig(VerifyAndUpdateConfig):
"FLASH_ATTN or TRITON_ATTN instead."
)
if attention_config.backend is None and not attention_config.use_non_causal:
# Only the sm_100 default backend order reads this flag (it puts
# FlashInfer first for causal models). Elsewhere Gemma4Config above
# keeps FlashInfer out: FA4 on every layer, else TRITON_ATTN.
attention_config.use_non_causal = True
logger.info(
"DiffusionGemma uses mixed causal/bidirectional attention "
"within a batch; setting use_non_causal=True to exclude "
"FlashInfer from auto-selection."
"DiffusionGemma mixes causal and bidirectional attention in "
"one batch; setting use_non_causal=True so the default backend "
"order prefers FlashAttention over FlashInfer."
)
# Auto-create DiffusionConfig from HF config if not provided.