[Bugfix][Gemma 4] Clamp soft-token estimate to max_soft_tokens (#40796)
Signed-off-by: Hoang Nguyen <118159510+hnt2601@users.noreply.github.com> Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
This commit is contained in:
co-authored by
Claude
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Isotr0py
parent
8586369f61
commit
c3ad791e1a
@@ -265,7 +265,14 @@ class Gemma4ProcessingInfo(BaseProcessingInfo):
|
||||
target_h = max(unit, int(math.floor(image_height * scale / unit)) * unit)
|
||||
target_w = max(unit, int(math.floor(image_width * scale / unit)) * unit)
|
||||
num_patches = (target_h // patch_size) * (target_w // patch_size)
|
||||
return num_patches // (pooling_kernel_size**2)
|
||||
# Clamp to ``max_soft_tokens``: extreme aspect ratios (e.g. 3x900)
|
||||
# cause the floor() above to round one dim up to ``unit`` while the
|
||||
# other scales freely, which over-shoots ``max_patches``. The HF
|
||||
# Gemma 4 image processor caps its vision-tower output at
|
||||
# ``max_soft_tokens``, so without this clamp the prompt-side
|
||||
# placeholder count exceeds the encoder output and
|
||||
# ``_merge_multimodal_embeddings`` crashes.
|
||||
return min(num_patches // (pooling_kernel_size**2), max_soft_tokens)
|
||||
|
||||
def get_image_repl(
|
||||
self,
|
||||
|
||||
Reference in New Issue
Block a user