[Docs] Fix the typos in the document (#57190)
Co-authored-by: Young <yangchh15@163.com>
This commit is contained in:
@@ -147,7 +147,7 @@ With standard decoding, each streamed output usually contains one token, so ITL
|
||||
and TPOT are typically similar.
|
||||
|
||||
With speculative decoding, one streamed output can contain multiple tokens,
|
||||
such as several accepted draft tokens within a single engine tstep. ITL records
|
||||
such as several accepted draft tokens within a single engine step. ITL records
|
||||
only the gaps between streamed outputs; it does not add zero-duration gaps for
|
||||
tokens in the same output. TPOT instead amortizes the request's decoding time
|
||||
over every output token.
|
||||
@@ -451,7 +451,7 @@ vllm bench serve \
|
||||
--num-prompts -1
|
||||
```
|
||||
|
||||
Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructued data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card.
|
||||
Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructured data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card.
|
||||
|
||||
#### BFCL (Tool-Calling) Benchmark
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ When run, benchmark script generates results under **benchmark/results** folder,
|
||||
- `ON_CPU`: set the value to '1' on Intel® Xeon® and Arm® Neoverse™ Processors. Default value is 0.
|
||||
- `SERVING_JSON`: JSON file to use for the serving tests. Default value is empty string (use default file).
|
||||
- `LATENCY_JSON`: JSON file to use for the latency tests. Default value is empty string (use default file).
|
||||
- `THROUGHPUT_JSON`: JSON file to use for the throughout tests. Default value is empty string (use default file).
|
||||
- `THROUGHPUT_JSON`: JSON file to use for the throughput tests. Default value is empty string (use default file).
|
||||
- `REMOTE_HOST`: IP for the remote vLLM service to benchmark. Default value is empty string.
|
||||
- `REMOTE_PORT`: Port for the remote vLLM service to benchmark. Default value is empty string.
|
||||
- `PROMPTS_PER_CONCURRENCY`: Multiplier to compute `num_prompts` for serving tests (`num_prompts = max_concurrency × value`). Overrides JSON `num_prompts`. Default is NULL.
|
||||
|
||||
@@ -50,7 +50,7 @@ To rebuild only the Rust frontend binary:
|
||||
./build_rust.sh --debug # faster build for development
|
||||
```
|
||||
|
||||
If you are developing vLLM's Python and CUDA/C++ code, install Pytorch first:
|
||||
If you are developing vLLM's Python and CUDA/C++ code, install PyTorch first:
|
||||
|
||||
```bash
|
||||
uv pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu129
|
||||
|
||||
@@ -670,7 +670,7 @@ fall under the more general heading of "Observability".
|
||||
vLLM has support for OpenTelemetry tracing:
|
||||
|
||||
- Added by <https://github.com/vllm-project/vllm/pull/4687> and reinstated by <https://github.com/vllm-project/vllm/pull/20372>
|
||||
- Configured with `--oltp-traces-endpoint` and `--collect-detailed-traces`
|
||||
- Configured with `--otlp-traces-endpoint` and `--collect-detailed-traces`
|
||||
- [OpenTelemetry blog post](https://opentelemetry.io/blog/2024/llm-observability/)
|
||||
- [User-facing docs](../../examples/observability/opentelemetry/README.md)
|
||||
- [Blog post](https://medium.com/@ronen.schaffer/follow-the-trail-supercharging-vllm-with-opentelemetry-distributed-tracing-aa655229b46f)
|
||||
|
||||
@@ -100,7 +100,7 @@ kv-cache and the quantization algorithm is AutoSmoothQuant.
|
||||
|
||||
!!! note
|
||||
Note the quantization algorithm needs a JSON config file and the config file is located in
|
||||
[Quark Pytorch examples](https://quark.docs.amd.com/latest/pytorch/pytorch_examples.html),
|
||||
[Quark PyTorch examples](https://quark.docs.amd.com/latest/pytorch/pytorch_examples.html),
|
||||
under the directory `examples/torch/language_modeling/llm_ptq/models`. For example,
|
||||
AutoSmoothQuant config file for Llama is
|
||||
`examples/torch/language_modeling/llm_ptq/models/llama/autosmoothquant_config.json`.
|
||||
|
||||
@@ -96,7 +96,7 @@ The code used to request completions as a client remains unchanged:
|
||||
},
|
||||
gpu_memory_utilization=0.5,
|
||||
)
|
||||
outputs = llm.generate(prompts,sampling_params)
|
||||
outputs = llm.generate(prompts, sampling_params)
|
||||
|
||||
for output in outputs:
|
||||
prompt = output.prompt
|
||||
|
||||
@@ -33,7 +33,7 @@ uv pip install "https://github.com/vllm-project/vllm/releases/download/v${VLLM_V
|
||||
pip install "https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cpu-cp38-abi3-manylinux_2_34_x86_64.whl" --extra-index-url https://download.pytorch.org/whl/cpu
|
||||
```
|
||||
!!! warning "set `LD_PRELOAD`"
|
||||
Before use vLLM CPU installed via wheels, make Intel OpenMP is added to `LD_PRELOAD`:
|
||||
Before use vLLM CPU installed via wheels, make sure Intel OpenMP is added to `LD_PRELOAD`:
|
||||
```bash
|
||||
# manually find the path
|
||||
sudo find / -iname *libiomp5.so
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
This document lists the current committers of the vLLM project and the core areas they maintain.
|
||||
Committers have write access to the vLLM repository and are responsible for reviewing and merging PRs.
|
||||
You can also refer to the [CODEOWNERS](https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS) file for concrete file-level ownership and reviewers. Both this documents and the CODEOWNERS file are living documents and they complement each other.
|
||||
You can also refer to the [CODEOWNERS](https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS) file for concrete file-level ownership and reviewers. Both these documents and the CODEOWNERS file are living documents and they complement each other.
|
||||
|
||||
## Active Committers
|
||||
|
||||
|
||||
@@ -233,7 +233,7 @@ Two implementations ship with vLLM:
|
||||
| `RayVLLMWeightSyncClient(handle)` | One or more `AsyncLLM`/`LLM` Ray actors. Accepts a list and fans each call out to every handle, blocking on all of them, so a multi-actor (e.g. multi-DP) deployment is driven as one unit |
|
||||
| `HTTPVLLMWeightSyncClient(base_url, timeout=300)` | A vLLM server over the RLHF HTTP routes |
|
||||
|
||||
Custom weight sync clients can be implement like so:
|
||||
Custom weight sync clients can be implemented like so:
|
||||
|
||||
```python
|
||||
class MyFrameworkWeightSyncClient:
|
||||
|
||||
Reference in New Issue
Block a user