[Misc] fix line length for entire codebase (#3444)
This commit is contained in:
@@ -37,9 +37,10 @@ for output in outputs:
|
||||
|
||||
print("-" * 80)
|
||||
|
||||
# The llm.generate call will batch all prompts and send the batch at once if resources allow.
|
||||
# The prefix will only be cached after the first batch is processed, so we need to call generate once
|
||||
# to calculate the prefix and cache it.
|
||||
# The llm.generate call will batch all prompts and send the batch at once
|
||||
# if resources allow. The prefix will only be cached after the first batch
|
||||
# is processed, so we need to call generate once to calculate the prefix
|
||||
# and cache it.
|
||||
outputs = llm.generate(generating_prompts[0], sampling_params)
|
||||
|
||||
# Subsequent batches can leverage the cached prefix
|
||||
|
||||
Reference in New Issue
Block a user