diff --git a/docs/benchmarking/cli.md b/docs/benchmarking/cli.md index d46ab04b0e..e7ee794da6 100644 --- a/docs/benchmarking/cli.md +++ b/docs/benchmarking/cli.md @@ -147,7 +147,7 @@ With standard decoding, each streamed output usually contains one token, so ITL and TPOT are typically similar. With speculative decoding, one streamed output can contain multiple tokens, -such as several accepted draft tokens within a single engine tstep. ITL records +such as several accepted draft tokens within a single engine step. ITL records only the gaps between streamed outputs; it does not add zero-duration gaps for tokens in the same output. TPOT instead amortizes the request's decoding time over every output token. @@ -451,7 +451,7 @@ vllm bench serve \ --num-prompts -1 ``` -Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructued data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card. +Available categories include `[high_entropy, mixed, low_entropy]`, where high entropy data contains unstructured data such as creative writing while low entropy data contains more structured data such as coding, more details are in the dataset card. #### BFCL (Tool-Calling) Benchmark diff --git a/docs/benchmarking/dashboard.md b/docs/benchmarking/dashboard.md index 44effc078e..4c1132c17f 100644 --- a/docs/benchmarking/dashboard.md +++ b/docs/benchmarking/dashboard.md @@ -36,7 +36,7 @@ When run, benchmark script generates results under **benchmark/results** folder, - `ON_CPU`: set the value to '1' on Intel® Xeon® and Arm® Neoverse™ Processors. Default value is 0. - `SERVING_JSON`: JSON file to use for the serving tests. Default value is empty string (use default file). - `LATENCY_JSON`: JSON file to use for the latency tests. Default value is empty string (use default file). -- `THROUGHPUT_JSON`: JSON file to use for the throughout tests. Default value is empty string (use default file). +- `THROUGHPUT_JSON`: JSON file to use for the throughput tests. Default value is empty string (use default file). - `REMOTE_HOST`: IP for the remote vLLM service to benchmark. Default value is empty string. - `REMOTE_PORT`: Port for the remote vLLM service to benchmark. Default value is empty string. - `PROMPTS_PER_CONCURRENCY`: Multiplier to compute `num_prompts` for serving tests (`num_prompts = max_concurrency × value`). Overrides JSON `num_prompts`. Default is NULL. diff --git a/docs/contributing/README.md b/docs/contributing/README.md index 7a1af9c1a7..e4b86f99ae 100644 --- a/docs/contributing/README.md +++ b/docs/contributing/README.md @@ -50,7 +50,7 @@ To rebuild only the Rust frontend binary: ./build_rust.sh --debug # faster build for development ``` -If you are developing vLLM's Python and CUDA/C++ code, install Pytorch first: +If you are developing vLLM's Python and CUDA/C++ code, install PyTorch first: ```bash uv pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu129 diff --git a/docs/design/metrics.md b/docs/design/metrics.md index 3d108890d7..3e070f5888 100644 --- a/docs/design/metrics.md +++ b/docs/design/metrics.md @@ -670,7 +670,7 @@ fall under the more general heading of "Observability". vLLM has support for OpenTelemetry tracing: - Added by and reinstated by -- Configured with `--oltp-traces-endpoint` and `--collect-detailed-traces` +- Configured with `--otlp-traces-endpoint` and `--collect-detailed-traces` - [OpenTelemetry blog post](https://opentelemetry.io/blog/2024/llm-observability/) - [User-facing docs](../../examples/observability/opentelemetry/README.md) - [Blog post](https://medium.com/@ronen.schaffer/follow-the-trail-supercharging-vllm-with-opentelemetry-distributed-tracing-aa655229b46f) diff --git a/docs/features/quantization/quark.md b/docs/features/quantization/quark.md index 8445288ff9..e5cfb8653b 100644 --- a/docs/features/quantization/quark.md +++ b/docs/features/quantization/quark.md @@ -100,7 +100,7 @@ kv-cache and the quantization algorithm is AutoSmoothQuant. !!! note Note the quantization algorithm needs a JSON config file and the config file is located in - [Quark Pytorch examples](https://quark.docs.amd.com/latest/pytorch/pytorch_examples.html), + [Quark PyTorch examples](https://quark.docs.amd.com/latest/pytorch/pytorch_examples.html), under the directory `examples/torch/language_modeling/llm_ptq/models`. For example, AutoSmoothQuant config file for Llama is `examples/torch/language_modeling/llm_ptq/models/llama/autosmoothquant_config.json`. diff --git a/docs/features/speculative_decoding/draft_model.md b/docs/features/speculative_decoding/draft_model.md index 5ab4629409..880ea1df59 100644 --- a/docs/features/speculative_decoding/draft_model.md +++ b/docs/features/speculative_decoding/draft_model.md @@ -96,7 +96,7 @@ The code used to request completions as a client remains unchanged: }, gpu_memory_utilization=0.5, ) -outputs = llm.generate(prompts,sampling_params) +outputs = llm.generate(prompts, sampling_params) for output in outputs: prompt = output.prompt diff --git a/docs/getting_started/installation/cpu.x86.inc.md b/docs/getting_started/installation/cpu.x86.inc.md index a64d0e5703..260ffe2863 100644 --- a/docs/getting_started/installation/cpu.x86.inc.md +++ b/docs/getting_started/installation/cpu.x86.inc.md @@ -33,7 +33,7 @@ uv pip install "https://github.com/vllm-project/vllm/releases/download/v${VLLM_V pip install "https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cpu-cp38-abi3-manylinux_2_34_x86_64.whl" --extra-index-url https://download.pytorch.org/whl/cpu ``` !!! warning "set `LD_PRELOAD`" - Before use vLLM CPU installed via wheels, make Intel OpenMP is added to `LD_PRELOAD`: + Before use vLLM CPU installed via wheels, make sure Intel OpenMP is added to `LD_PRELOAD`: ```bash # manually find the path sudo find / -iname *libiomp5.so diff --git a/docs/governance/committers.md b/docs/governance/committers.md index bd36759122..48ed4b2607 100644 --- a/docs/governance/committers.md +++ b/docs/governance/committers.md @@ -2,7 +2,7 @@ This document lists the current committers of the vLLM project and the core areas they maintain. Committers have write access to the vLLM repository and are responsible for reviewing and merging PRs. -You can also refer to the [CODEOWNERS](https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS) file for concrete file-level ownership and reviewers. Both this documents and the CODEOWNERS file are living documents and they complement each other. +You can also refer to the [CODEOWNERS](https://github.com/vllm-project/vllm/blob/main/.github/CODEOWNERS) file for concrete file-level ownership and reviewers. Both these documents and the CODEOWNERS file are living documents and they complement each other. ## Active Committers diff --git a/docs/training/weight_transfer/base.md b/docs/training/weight_transfer/base.md index d73c88436a..759c55ef6f 100644 --- a/docs/training/weight_transfer/base.md +++ b/docs/training/weight_transfer/base.md @@ -233,7 +233,7 @@ Two implementations ship with vLLM: | `RayVLLMWeightSyncClient(handle)` | One or more `AsyncLLM`/`LLM` Ray actors. Accepts a list and fans each call out to every handle, blocking on all of them, so a multi-actor (e.g. multi-DP) deployment is driven as one unit | | `HTTPVLLMWeightSyncClient(base_url, timeout=300)` | A vLLM server over the RLHF HTTP routes | -Custom weight sync clients can be implement like so: +Custom weight sync clients can be implemented like so: ```python class MyFrameworkWeightSyncClient: