[Model Runner V2][MM] Support EVS (#46535)

Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Nick Hill
2026-06-24 10:18:56 -04:00
committed by GitHub
co-authored by mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent 061043eaca
commit 84c62e1cbd
10 changed files with 245 additions and 50 deletions
+6 -5
View File
@@ -421,11 +421,11 @@ class SupportsMultiModalPruning(Protocol):
def recompute_mrope_positions(
self,
input_ids: list[int],
multimodal_embeddings: MultiModalEmbeddings,
input_ids: list[int] | torch.Tensor,
multimodal_embeddings: Sequence[torch.Tensor],
mrope_positions: torch.LongTensor,
num_computed_tokens: int,
) -> tuple[MultiModalEmbeddings, Tensor, int]:
) -> tuple[Sequence[torch.Tensor], Tensor, int]:
"""
Update part of input mrope positions (starting with
num_computed_tokens index). Original mrope_positions are computed
@@ -435,8 +435,9 @@ class SupportsMultiModalPruning(Protocol):
Args:
input_ids: (N,) All input tokens of the prompt containing
entire sequence.
multimodal_embeddings: Tuple of multimodal embeddings that
entire sequence. Either a host-side list or an already
device-resident tensor.
multimodal_embeddings: Sequence of multimodal embeddings that
fits into the prefill chunk that is being processed.
mrope_positions: Existing mrope positions (3, N) for entire
sequence