Generative Scoring (#34539)

Signed-off-by: Vedant Jhaveri <vjhaveri@linkedin.com>
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
This commit is contained in:
Vedant V Jhaveri
2026-03-31 16:02:11 -07:00
committed by GitHub
co-authored by Vedant Jhaveri Claude Opus 4.6 Cyrus Leung
parent 36f1dc19ae
commit 2e56975657
13 changed files with 1265 additions and 3 deletions
+6
View File
@@ -232,6 +232,12 @@ class SamplingParams(
prompt_logprobs: int | None = None
"""Number of log probabilities to return per prompt token.
When set to -1, return all `vocab_size` log probabilities."""
logprob_token_ids: list[int] | None = None
"""Specific token IDs to return logprobs for. More efficient than
logprobs=-1 when you only need logprobs for a small set of tokens.
When set, logprobs for exactly these token IDs will be returned,
in addition to the sampled token. This is useful for scoring tasks
where you want to compare probabilities of specific label tokens."""
flat_logprobs: bool = False
"""Whether to return logprobs in flatten format (i.e. FlatLogprob)
for better performance.