[ROCM][Bugfix] Deferring to TRITON_ATTN backend for Prefix lm models instead of ROCM_ATTN and fixed minor bug in Gemma3 Multimodal code (#46754)
Signed-off-by: GodRishUniverse <rishabh.agarwal@amd.com> Signed-off-by: Rishabh Agarwal <risagarw@amd.com> Signed-off-by: GodRishUniverse <risagarw@amd.com>
This commit is contained in:
@@ -557,6 +557,10 @@ class Gemma3ForConditionalGeneration(
|
||||
)
|
||||
self.multi_modal_projector = Gemma3MultiModalProjector(config)
|
||||
|
||||
# https://huggingface.co/blog/gemma3, (image tokens attend to each other)
|
||||
# explicit mention here as text side in gemma3 standalone does not need
|
||||
# prefix_lm but multimodal does
|
||||
config.text_config.is_mm_prefix_lm = True
|
||||
with self._mark_language_model(vllm_config):
|
||||
self.language_model = init_vllm_registered_model(
|
||||
vllm_config=vllm_config,
|
||||
|
||||
Reference in New Issue
Block a user