[V1] Initial support of multimodal models for V1 re-arch (#10699)
Signed-off-by: Roger Wang <ywang@roblox.com>
This commit is contained in:
@@ -36,6 +36,11 @@ class SupportsMultiModal(Protocol):
|
||||
"""
|
||||
Returns multimodal embeddings generated from multimodal kwargs
|
||||
to be merged with text embeddings.
|
||||
|
||||
The output embeddings must be one of the following formats:
|
||||
- A list or tuple of 2D tensors, where each tensor corresponds to
|
||||
each input image.
|
||||
- A single 3D tensor, with the batch dimension grouping the 2D tensors.
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
Reference in New Issue
Block a user