[Model Runner V2][MM] Support EVS (#46535)
Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
061043eaca
commit
84c62e1cbd
@@ -421,11 +421,11 @@ class SupportsMultiModalPruning(Protocol):
|
||||
|
||||
def recompute_mrope_positions(
|
||||
self,
|
||||
input_ids: list[int],
|
||||
multimodal_embeddings: MultiModalEmbeddings,
|
||||
input_ids: list[int] | torch.Tensor,
|
||||
multimodal_embeddings: Sequence[torch.Tensor],
|
||||
mrope_positions: torch.LongTensor,
|
||||
num_computed_tokens: int,
|
||||
) -> tuple[MultiModalEmbeddings, Tensor, int]:
|
||||
) -> tuple[Sequence[torch.Tensor], Tensor, int]:
|
||||
"""
|
||||
Update part of input mrope positions (starting with
|
||||
num_computed_tokens index). Original mrope_positions are computed
|
||||
@@ -435,8 +435,9 @@ class SupportsMultiModalPruning(Protocol):
|
||||
|
||||
Args:
|
||||
input_ids: (N,) All input tokens of the prompt containing
|
||||
entire sequence.
|
||||
multimodal_embeddings: Tuple of multimodal embeddings that
|
||||
entire sequence. Either a host-side list or an already
|
||||
device-resident tensor.
|
||||
multimodal_embeddings: Sequence of multimodal embeddings that
|
||||
fits into the prefill chunk that is being processed.
|
||||
mrope_positions: Existing mrope positions (3, N) for entire
|
||||
sequence
|
||||
|
||||
Reference in New Issue
Block a user