[Model] Support EmbeddingGemma2 multimodal pooling architecture (#60254)
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com> Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
This commit is contained in:
co-authored by
Luciano Martins
parent
7b665f7a58
commit
02b83919aa
@@ -243,9 +243,10 @@ def context_attention_fwd(
|
||||
b_seq_len: [b]
|
||||
out: [b * s, head, head_dim]
|
||||
"""
|
||||
BLOCK = get_block_size(q.dtype)
|
||||
|
||||
Lq, Lk, _ = q.shape[-1], k.shape[-1], v.shape[-1]
|
||||
BLOCK = get_block_size(q.dtype)
|
||||
if Lk >= 512:
|
||||
BLOCK = min(BLOCK, 32)
|
||||
|
||||
sm_scale = 1.0 / (Lq**0.5) if softmax_scale is None else softmax_scale
|
||||
# rescale with 1/ln(2) for triton exp2
|
||||
|
||||
Reference in New Issue
Block a user