[bugfix] Fix prompt logprobs on request eviction during chunked prefill (#41411)
Signed-off-by: Joachim Studnia <joachim@mistral.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
8c780943b4
commit
422dd02598
@@ -996,6 +996,8 @@ class VllmRunner:
|
||||
req_sample_output_ids: list[list[int]] = []
|
||||
req_sample_output_strs: list[str] = []
|
||||
req_logprobs = []
|
||||
if req_output.prompt_logprobs:
|
||||
req_logprobs.extend(req_output.prompt_logprobs)
|
||||
for sample in req_output.outputs:
|
||||
output_str = sample.text
|
||||
output_ids = list(sample.token_ids)
|
||||
|
||||
Reference in New Issue
Block a user