[Entrypoints] Split the pooling offline API into PoolingOfflineMixin. (#42267)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
This commit is contained in:
wang.yuqi
2026-05-15 08:05:57 +00:00
committed by GitHub
parent 4b364f810e
commit 75fd68c7a5
9 changed files with 531 additions and 439 deletions
+3 -3
View File
@@ -120,7 +120,7 @@ The following [pooling parameters][vllm.PoolingParams] are supported.
### `LLM.embed`
The [embed][vllm.LLM.embed] method outputs an embedding vector for each prompt.
The [embed][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.embed] method outputs an embedding vector for each prompt.
```python
from vllm import LLM
@@ -136,7 +136,7 @@ A code example can be found here: [examples/basic/offline_inference/embed.py](..
### `LLM.encode`
The [encode][vllm.LLM.encode] method is available to all pooling models in vLLM.
The [encode][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.encode] method is available to all pooling models in vLLM.
Set `pooling_task="embed"` when using `LLM.encode` for embedding Models:
@@ -152,7 +152,7 @@ print(f"Data: {data!r}")
### `LLM.score`
The [score][vllm.LLM.score] method outputs similarity scores between sentence pairs.
The [score][vllm.entrypoints.pooling.offline.PoolingOfflineMixin.score] method outputs similarity scores between sentence pairs.
All models that support embedding task also support using the score API to compute similarity scores by calculating the cosine similarity of two input prompt's embeddings.