[Core][Model] Gemma4: Unified FA4 for all layers + FlashAttention mm_prefix support (#42175)

Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
This commit is contained in:
Luciano Martins
2026-06-09 17:45:39 -07:00
committed by GitHub
co-authored by Luciano Martins Lucas Wilkinson Matthew Bonanni
parent d82ac00923
commit 6deb05e0e4
14 changed files with 222 additions and 142 deletions
+5
View File
@@ -73,6 +73,11 @@ class DFlashProposer(SpecDecodeBaseProposer):
@override
def _create_draft_vllm_config(self) -> VllmConfig:
base = super()._create_draft_vllm_config()
# The draft model is text-only — clear the target's multimodal
# flag so flash_attn is not rejected for mm_prefix support.
arch = base.model_config.model_arch_config
if arch.is_mm_prefix_lm:
base.model_config.model_arch_config = replace(arch, is_mm_prefix_lm=False)
return replace(
base,
attention_config=replace(