[Bugfix][Frontend] Honor the constructor chat template in offline scoring (#61990)

Signed-off-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
Co-authored-by: mottopanikeiku <176798723+mottopanikeiku@users.noreply.github.com>
This commit is contained in:
alp
2026-10-16 10:20:46 +00:00
committed by GitHub
co-authored by mottopanikeiku
parent 024c5188bf
commit 4ab99ff136
4 changed files with 55 additions and 4 deletions
+2
View File
@@ -455,6 +455,8 @@ Score templates are supported for **cross-encoder** models only. If you are usin
Some scoring models require a specific prompt format to work correctly. You can specify a custom score template using the `--chat-template` parameter (see [Chat Template](../../serving/online_serving/README.md#chat-template)).
For offline scoring, pass `chat_template` to the `LLM` constructor to use it for every `LLM.score` call. A `chat_template` passed to `LLM.score` overrides the constructor's template for that call.
Like chat templates, the score template receives a `messages` list. For scoring, each message has a `role` attribute—either `"query"` or `"document"`. For the usual kind of point-wise cross-encoder, you can expect exactly two messages: one query and one document. To access the query and document content, use Jinja's `selectattr` filter:
- **Query**: `{{ (messages | selectattr("role", "eq", "query") | first).content }}`