[Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
This commit is contained in:
wang.yuqi
2026-05-15 08:05:57 +00:00
committed by GitHub
parent 4b364f810e
commit 75fd68c7a5
9 changed files with 531 additions and 439 deletions
+4 -4
View File
@@ -131,24 +131,24 @@ enabling the corresponding APIs.
### `LLM.classify`
The [classify][vllm.LLM.classify] method outputs a probability vector for each prompt.
The [classify][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.classify] method outputs a probability vector for each prompt.
It is primarily designed for [classification models](classify.md).
For more information about `LLM.embed`, see [this page](classify.md#offline-inference).
### `LLM.embed`
The [embed][vllm.LLM.embed] method outputs an embedding vector for each prompt.
The [embed][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.embed] method outputs an embedding vector for each prompt.
It is primarily designed for [embedding models](embed.md).
For more information about `LLM.embed`, see [this page](embed.md#offline-inference).
### `LLM.score`
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
It is primarily designed for [score models](scoring.md).
### `LLM.encode`
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
Please use one of the more specific methods or set the task directly when using `LLM.encode`, refer to the [table above](#offline-apis-corresponding-to-pooling-usages).
+2 -2
View File
@@ -77,7 +77,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
### `LLM.classify`
The [classify][vllm.LLM.classify] method outputs a probability vector for each prompt.
The [classify][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.classify] method outputs a probability vector for each prompt.
```python
from vllm import LLM
@@ -93,7 +93,7 @@ A code example can be found here: [examples/basic/offline_inference/classify.py]
### `LLM.encode`
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
Set `pooling_task="classify"` when using `LLM.encode` for classification Models:
+3 -3
View File
@@ -120,7 +120,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
### `LLM.embed`
The [embed][vllm.LLM.embed] method outputs an embedding vector for each prompt.
The [embed][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.embed] method outputs an embedding vector for each prompt.
```python
from vllm import LLM
@@ -136,7 +136,7 @@ A code example can be found here: [examples/basic/offline_inference/embed.py](..
### `LLM.encode`
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
Set `pooling_task="embed"` when using `LLM.encode` for embedding Models:
@@ -152,7 +152,7 @@ print(f"Data: {data!r}")
### `LLM.score`
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
All models that support embedding task also support using the score API to compute similarity scores by calculating the cosine similarity of two input prompt's embeddings.
+1 -1
View File
@@ -87,7 +87,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
### `LLM.encode`
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
- Reward Models
+1 -1
View File
@@ -130,7 +130,7 @@ The following [pooling parameters][vllm.PoolingParams] are only supported by cro
### `LLM.score`
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
```python
from vllm import LLM
+1 -1
View File
@@ -87,7 +87,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
### `LLM.encode`
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
Set `pooling_task="token_classify"` when using `LLM.encode` for token classification Models:
+2 -2
View File
@@ -94,7 +94,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
### `LLM.encode`
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
Set `pooling_task="token_embed"` when using `LLM.encode` for token embedding Models:
@@ -110,7 +110,7 @@ print(f"Data: {data!r}")
### `LLM.score`
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
All models that support token embedding task also support using the score API to compute similarity scores by calculating the late interaction of two input prompts.