[Bugfix][V1/V2] Fix prompt_logprobs to respect logprobs_mode (#47680)

Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com>
Signed-off-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
Signed-off-by: Allen Shen <aoshen@inferact.ai>
Co-authored-by: Wojciech Wais <wojciech.wais@gmail.com>
Co-authored-by: Federico Kamelhar <209537060+fede-kamel@users.noreply.github.com>
This commit is contained in:
aoshen02
2026-07-17 21:58:59 +01:00
committed by GitHub
co-authored by Wojciech Wais Federico Kamelhar
parent 088c0be268
commit 41ea2dd44a
11 changed files with 106 additions and 40 deletions
+4 -1
View File
@@ -341,7 +341,10 @@ class GPUModelRunner(LoRAModelRunnerMixin):
self.speculative_config,
self.device,
)
self.prompt_logprobs_worker = PromptLogprobsWorker(self.max_num_reqs)
self.prompt_logprobs_worker = PromptLogprobsWorker(
self.max_num_reqs,
logprobs_mode=self.model_config.logprobs_mode,
)
self.structured_outputs_worker = StructuredOutputsWorker(
max_num_logits=self.max_num_reqs * self.decode_query_len,
vocab_size=self.vocab_size,