[Bugfix] Support Sentence Transformers 5.4+ serialized configs (#54533)

Signed-off-by: maireneu <maireneu@gmail.com>
Co-authored-by: Codex <codex@openai.com>
This commit is contained in:
Eli Nabro Yeongmin Oh
2026-09-01 15:53:38 +00:00
committed by GitHub
co-authored by Codex
parent f1e5fdd7f2
commit cdefd9d499
3 changed files with 150 additions and 21 deletions
+3 -3
View File
@@ -292,12 +292,12 @@ classification activation is applied.
| Field | Source precedence | How to override |
| ----- | ----------------- | --------------- |
| Pooling method (`pooling_type`) | `--pooler-config` > boolean `pooling_mode_*` fields in the Pooling module referenced by Sentence Transformers `modules.json` > architecture default (`LAST` for sequence pooling and `ALL` for token pooling unless the architecture overrides it) | Set `{"pooling_type": "CLS"}`, or set `seq_pooling_type` / `tok_pooling_type` explicitly. |
| Pooling method (`pooling_type`) | `--pooler-config` > compact `pooling_mode` or legacy boolean `pooling_mode_*` fields in the Pooling module referenced by Sentence Transformers `modules.json` > architecture default (`LAST` for sequence pooling and `ALL` for token pooling unless the architecture overrides it) | Set `{"pooling_type": "CLS"}`, or set `seq_pooling_type` / `tok_pooling_type` explicitly. |
| Embedding normalization (`use_activation`) | `--pooler-config` > Sentence Transformers modules (`true` when a Normalize module is present, otherwise `false`) > pooling-task default (`true`) when no Sentence Transformers Pooling module is found | Set `{"use_activation": false}` to return unnormalized embeddings. |
| Classification activation function | Hugging Face `problem_type` > Sentence Transformers activation metadata > sigmoid or softmax selected from the label count | The function cannot be selected through `--pooler-config`; set `{"use_activation": false}` to return logits instead. |
Sentence Transformers configurations using the newer compact `pooling_mode`
string are not currently parsed; see [issue #45995](https://github.com/vllm-project/vllm/issues/45995).
Both compact `pooling_mode` values written by Sentence Transformers 5.4+ and
legacy boolean `pooling_mode_*` fields are supported.
For converted models and predefined models using the standard DispatchPooler
adapters, `embed` and `token_embed` construct an L2-normalization head, while