[Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
This commit is contained in:
@@ -131,24 +131,24 @@ enabling the corresponding APIs.
|
||||
|
||||
### `LLM.classify`
|
||||
|
||||
The [classify][vllm.LLM.classify] method outputs a probability vector for each prompt.
|
||||
The [classify][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.classify] method outputs a probability vector for each prompt.
|
||||
It is primarily designed for [classification models](classify.md).
|
||||
For more information about `LLM.embed`, see [this page](classify.md#offline-inference).
|
||||
|
||||
### `LLM.embed`
|
||||
|
||||
The [embed][vllm.LLM.embed] method outputs an embedding vector for each prompt.
|
||||
The [embed][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.embed] method outputs an embedding vector for each prompt.
|
||||
It is primarily designed for [embedding models](embed.md).
|
||||
For more information about `LLM.embed`, see [this page](embed.md#offline-inference).
|
||||
|
||||
### `LLM.score`
|
||||
|
||||
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
|
||||
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
|
||||
It is primarily designed for [score models](scoring.md).
|
||||
|
||||
### `LLM.encode`
|
||||
|
||||
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
|
||||
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
|
||||
|
||||
Please use one of the more specific methods or set the task directly when using `LLM.encode`, refer to the [table above](#offline-apis-corresponding-to-pooling-usages).
|
||||
|
||||
|
||||
@@ -77,7 +77,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
|
||||
|
||||
### `LLM.classify`
|
||||
|
||||
The [classify][vllm.LLM.classify] method outputs a probability vector for each prompt.
|
||||
The [classify][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.classify] method outputs a probability vector for each prompt.
|
||||
|
||||
```python
|
||||
from vllm import LLM
|
||||
@@ -93,7 +93,7 @@ A code example can be found here: [examples/basic/offline_inference/classify.py]
|
||||
|
||||
### `LLM.encode`
|
||||
|
||||
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
|
||||
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
|
||||
|
||||
Set `pooling_task="classify"` when using `LLM.encode` for classification Models:
|
||||
|
||||
|
||||
@@ -120,7 +120,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
|
||||
|
||||
### `LLM.embed`
|
||||
|
||||
The [embed][vllm.LLM.embed] method outputs an embedding vector for each prompt.
|
||||
The [embed][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.embed] method outputs an embedding vector for each prompt.
|
||||
|
||||
```python
|
||||
from vllm import LLM
|
||||
@@ -136,7 +136,7 @@ A code example can be found here: [examples/basic/offline_inference/embed.py](..
|
||||
|
||||
### `LLM.encode`
|
||||
|
||||
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
|
||||
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
|
||||
|
||||
Set `pooling_task="embed"` when using `LLM.encode` for embedding Models:
|
||||
|
||||
@@ -152,7 +152,7 @@ print(f"Data: {data!r}")
|
||||
|
||||
### `LLM.score`
|
||||
|
||||
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
|
||||
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
|
||||
|
||||
All models that support embedding task also support using the score API to compute similarity scores by calculating the cosine similarity of two input prompt's embeddings.
|
||||
|
||||
|
||||
@@ -87,7 +87,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
|
||||
|
||||
### `LLM.encode`
|
||||
|
||||
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
|
||||
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
|
||||
|
||||
- Reward Models
|
||||
|
||||
|
||||
@@ -130,7 +130,7 @@ The following [pooling parameters][vllm.PoolingParams] are only supported by cro
|
||||
|
||||
### `LLM.score`
|
||||
|
||||
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
|
||||
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
|
||||
|
||||
```python
|
||||
from vllm import LLM
|
||||
|
||||
@@ -87,7 +87,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
|
||||
|
||||
### `LLM.encode`
|
||||
|
||||
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
|
||||
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
|
||||
|
||||
Set `pooling_task="token_classify"` when using `LLM.encode` for token classification Models:
|
||||
|
||||
|
||||
@@ -94,7 +94,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
|
||||
|
||||
### `LLM.encode`
|
||||
|
||||
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
|
||||
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
|
||||
|
||||
Set `pooling_task="token_embed"` when using `LLM.encode` for token embedding Models:
|
||||
|
||||
@@ -110,7 +110,7 @@ print(f"Data: {data!r}")
|
||||
|
||||
### `LLM.score`
|
||||
|
||||
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
|
||||
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
|
||||
|
||||
All models that support token embedding task also support using the score API to compute similarity scores by calculating the late interaction of two input prompts.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user