[Misc] fix line length for entire codebase (#3444)

This commit is contained in:
Simon Mo
2024-03-16 00:36:29 -07:00
committed by GitHub
parent ad50bf4b25
commit 8e67598aa6
9 changed files with 174 additions and 128 deletions
+4 -3
View File
@@ -37,9 +37,10 @@ for output in outputs:
print("-" * 80)
# The llm.generate call will batch all prompts and send the batch at once if resources allow.
# The prefix will only be cached after the first batch is processed, so we need to call generate once
# to calculate the prefix and cache it.
# The llm.generate call will batch all prompts and send the batch at once
# if resources allow. The prefix will only be cached after the first batch
# is processed, so we need to call generate once to calculate the prefix
# and cache it.
outputs = llm.generate(generating_prompts[0], sampling_params)
# Subsequent batches can leverage the cached prefix