[Model Runner V2] Bug fix: logprob dtype int64/int32 issue (#41761)
Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Nick Hill
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
bbee532988
commit
d7af6b34d8
@@ -690,6 +690,14 @@ class SamplingParams(
|
||||
parameter="logprob_token_ids",
|
||||
value=n,
|
||||
)
|
||||
if self.logprobs is not None and self.logprobs != n:
|
||||
raise VLLMValidationError(
|
||||
f"When both logprobs and logprob_token_ids are set, "
|
||||
f"logprobs must equal len(logprob_token_ids). Got "
|
||||
f"logprobs={self.logprobs}, len(logprob_token_ids)={n}.",
|
||||
parameter="logprob_token_ids",
|
||||
value=n,
|
||||
)
|
||||
|
||||
# Validate prompt logprobs.
|
||||
if num_prompt_logprobs := self.prompt_logprobs:
|
||||
|
||||
Reference in New Issue
Block a user