[ROCM][Bugfix] Deferring to TRITON_ATTN backend for Prefix lm models instead of ROCM_ATTN and fixed minor bug in Gemma3 Multimodal code (#46754)

Signed-off-by: GodRishUniverse <rishabh.agarwal@amd.com>
Signed-off-by: Rishabh Agarwal <risagarw@amd.com>
Signed-off-by: GodRishUniverse <risagarw@amd.com>
This commit is contained in:
Rishabh Agarwal
2026-10-13 10:03:48 -05:00
committed by GitHub
parent f5fefbffe5
commit f395a6b1ad
2 changed files with 29 additions and 0 deletions
+4
View File
@@ -557,6 +557,10 @@ class Gemma3ForConditionalGeneration(
)
self.multi_modal_projector = Gemma3MultiModalProjector(config)
# https://huggingface.co/blog/gemma3, (image tokens attend to each other)
# explicit mention here as text side in gemma3 standalone does not need
# prefix_lm but multimodal does
config.text_config.is_mm_prefix_lm = True
with self._mark_language_model(vllm_config):
self.language_model = init_vllm_registered_model(
vllm_config=vllm_config,