Generative Scoring (#34539)
Signed-off-by: Vedant Jhaveri <vjhaveri@linkedin.com> Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
This commit is contained in:
co-authored by
Vedant Jhaveri
Claude Opus 4.6
Cyrus Leung
parent
36f1dc19ae
commit
2e56975657
@@ -232,6 +232,12 @@ class SamplingParams(
|
||||
prompt_logprobs: int | None = None
|
||||
"""Number of log probabilities to return per prompt token.
|
||||
When set to -1, return all `vocab_size` log probabilities."""
|
||||
logprob_token_ids: list[int] | None = None
|
||||
"""Specific token IDs to return logprobs for. More efficient than
|
||||
logprobs=-1 when you only need logprobs for a small set of tokens.
|
||||
When set, logprobs for exactly these token IDs will be returned,
|
||||
in addition to the sampled token. This is useful for scoring tasks
|
||||
where you want to compare probabilities of specific label tokens."""
|
||||
flat_logprobs: bool = False
|
||||
"""Whether to return logprobs in flatten format (i.e. FlatLogprob)
|
||||
for better performance.
|
||||
|
||||
Reference in New Issue
Block a user