Fix empty output when temp is too low (#2937)

Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
This commit is contained in:
Chang Su
2024-08-14 05:31:44 +00:00
committed by GitHub
co-authored by Cyrus Leung
parent 199adbb7cf
commit c134a46402
2 changed files with 9 additions and 1 deletions
+2 -1
View File
@@ -118,8 +118,9 @@ class Sampler(nn.Module):
sampling_tensors.frequency_penalties,
sampling_tensors.repetition_penalties)
# Apply temperature scaling.
# Use float32 to apply temperature scaling.
# Use in-place division to avoid creating a new tensor.
logits = logits.to(torch.float)
logits.div_(sampling_tensors.temperatures.unsqueeze(dim=1))
if do_top_p_top_k: