[Misc] Update benchmark_prefix_caching.py fixed example usage (#11920)
Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn> Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
This commit is contained in:
@@ -10,7 +10,8 @@ Fixed example usage:
|
|||||||
--model meta-llama/Llama-2-7b-chat-hf \
|
--model meta-llama/Llama-2-7b-chat-hf \
|
||||||
--enable-prefix-caching \
|
--enable-prefix-caching \
|
||||||
--num-prompts 1 \
|
--num-prompts 1 \
|
||||||
--repeat-count 100
|
--repeat-count 100 \
|
||||||
|
--input-length-range 128:256
|
||||||
|
|
||||||
ShareGPT example usage:
|
ShareGPT example usage:
|
||||||
# This command samples 20 prompts with input lengths
|
# This command samples 20 prompts with input lengths
|
||||||
|
|||||||
Reference in New Issue
Block a user