[Kernel][Model] Optimize FA4 mm_prefix range lookup (#50294)

Signed-off-by: Juqi Li <2223621784@qq.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Li Juqi
2026-08-05 07:33:49 -07:00
committed by GitHub
co-authored by Matthew Bonanni Claude Opus 5 mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 62c5e21621
commit 66b3c0e61f
5 changed files with 819 additions and 45 deletions
+2
View File
@@ -2078,6 +2078,8 @@ class Gemma4ForConditionalGeneration(
metadata.mm_prefix_range = None
if hasattr(metadata, "mm_prefix_range_tensor"):
metadata.mm_prefix_range_tensor = None
if hasattr(metadata, "mm_prefix_query_range_tensor"):
metadata.mm_prefix_query_range_tensor = None
if isinstance(attn_metadata, list):
for ub_metadata in attn_metadata: