[Frontend][last/5] Improve pooling entrypoints | clean up. (#39675)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
parent
5e5afafa21
commit
4e8c3f1c19
@@ -59,6 +59,16 @@ please refer to [IO Processor Plugins](../../design/io_processor_plugins.md).
|
||||
Within classification tasks, there is a specialized subcategory: Cross-encoder (aka reranker) models. These models
|
||||
are a subset of classification models that accept two prompts as input and output num_labels equal to 1.
|
||||
|
||||
### Pooling Types
|
||||
|
||||
| Pooling Tasks | Granularity | Description |
|
||||
|----------------|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| `CLS` pooling | Sequence-wise | For BERT‑like (bidirectional self‑attention) models, CLS pooling is used by default. This means the last_hidden_states corresponding to the first token (the [CLS] token) is taken as the output. |
|
||||
| `LAST` pooling | Sequence-wise | For GPT‑like (causal self‑attention) models, LAST pooling is used by default. This means the last_hidden_states corresponding to the last token is taken as the output. |
|
||||
| `MEAN` pooling | Sequence-wise | Many studies have shown that averaging the last_hidden_states over all input tokens performs better on certain downstream tasks. Therefore, more and more models are using MEAN pooling. |
|
||||
| `ALL` pooling | Token-wise | Outputs the last_hidden_states for all input tokens. |
|
||||
| `STEP` pooling | Token-wise | Filters and outputs the last_hidden_states corresponding to the token IDs returned by returned_token_ids. |
|
||||
|
||||
### Score Types
|
||||
|
||||
The scoring models is designed to compute similarity scores between two input prompts. It supports three model types
|
||||
|
||||
@@ -160,6 +160,8 @@ The following Score API parameters are supported:
|
||||
--8<-- "vllm/entrypoints/pooling/base/protocol.py:pooling-common-params"
|
||||
--8<-- "vllm/entrypoints/pooling/base/protocol.py:pooling-common-extra-params"
|
||||
--8<-- "vllm/entrypoints/pooling/base/protocol.py:classify-extra-params"
|
||||
--8<-- "vllm/entrypoints/pooling/scoring/protocol.py:scoring-common-params"
|
||||
--8<-- "vllm/entrypoints/pooling/scoring/protocol.py:score-request-params"
|
||||
```
|
||||
|
||||
#### Examples
|
||||
@@ -370,6 +372,8 @@ The following rerank api parameters are supported:
|
||||
--8<-- "vllm/entrypoints/pooling/base/protocol.py:pooling-common-params"
|
||||
--8<-- "vllm/entrypoints/pooling/base/protocol.py:pooling-common-extra-params"
|
||||
--8<-- "vllm/entrypoints/pooling/base/protocol.py:classify-extra-params"
|
||||
--8<-- "vllm/entrypoints/pooling/scoring/protocol.py:scoring-common-params"
|
||||
--8<-- "vllm/entrypoints/pooling/scoring/protocol.py:rerank-request-params"
|
||||
```
|
||||
|
||||
#### Examples
|
||||
|
||||
@@ -68,7 +68,7 @@ If your model is not in the above list, we will try to automatically convert the
|
||||
Forced alignment usage requires `--hf-overrides '{"architectures": ["Qwen3ASRForcedAlignerForTokenClassification"]}'`.
|
||||
Please refer to [examples/pooling/token_classify/forced_alignment_offline.py](../../../examples/pooling/token_classify/forced_alignment_offline.py).
|
||||
|
||||
### As Reward Models
|
||||
### Reward Models
|
||||
|
||||
Using token classification models as reward models. For details on reward models, see [Reward Models](reward.md).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user