[ModelRunner V2] Support encoder-only attention (#49331)

Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
This commit is contained in:
Nick Hill
2026-07-27 00:38:44 +00:00
committed by GitHub
co-authored by Taneem Ibrahim
parent 0934b26790
commit fdaa0d9e59
8 changed files with 276 additions and 4 deletions
+1 -1
View File
@@ -123,7 +123,7 @@ class BlockTables:
if self.num_kv_cache_groups == 1:
# Single group: write directly, skipping the per-write group lookup.
self.block_tables[0].apply_write()
else:
elif self.num_kv_cache_groups > 1:
# Multiple groups: apply all block tables with one fused kernel.
assert self.fused_writer is not None
self.fused_writer.apply(