[Docs] Fix the typos in the document (#57190)
Co-authored-by: Young <yangchh15@163.com>
This commit is contained in:
@@ -147,7 +147,7 @@ With standard decoding, each streamed output usually contains one token, so ITL
|
||||
and TPOT are typically similar.
|
||||
|
||||
With speculative decoding, one streamed output can contain multiple tokens,
|
||||
such as several accepted draft tokens within a single engine tstep. ITL records
|
||||
such as several accepted draft tokens within a single engine step. ITL records
|
||||
only the gaps between streamed outputs; it does not add zero-duration gaps for
|
||||
tokens in the same output. TPOT instead amortizes the request's decoding time
|
||||
over every output token.
|
||||
@@ -451,7 +451,7 @@ vllm bench serve \
|
||||
--num-prompts -1
|
||||
```
|
||||
|
||||
Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructued data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card.
|
||||
Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructured data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card.
|
||||
|
||||
#### BFCL (Tool-Calling) Benchmark
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ When run, benchmark script generates results under **benchmark/results** folder,
|
||||
- `ON_CPU`: set the value to '1' on Intel® Xeon® and Arm® Neoverse™ Processors. Default value is 0.
|
||||
- `SERVING_JSON`: JSON file to use for the serving tests. Default value is empty string (use default file).
|
||||
- `LATENCY_JSON`: JSON file to use for the latency tests. Default value is empty string (use default file).
|
||||
- `THROUGHPUT_JSON`: JSON file to use for the throughout tests. Default value is empty string (use default file).
|
||||
- `THROUGHPUT_JSON`: JSON file to use for the throughput tests. Default value is empty string (use default file).
|
||||
- `REMOTE_HOST`: IP for the remote vLLM service to benchmark. Default value is empty string.
|
||||
- `REMOTE_PORT`: Port for the remote vLLM service to benchmark. Default value is empty string.
|
||||
- `PROMPTS_PER_CONCURRENCY`: Multiplier to compute `num_prompts` for serving tests (`num_prompts = max_concurrency × value`). Overrides JSON `num_prompts`. Default is NULL.
|
||||
|
||||
Reference in New Issue
Block a user