From 529d0103511429f0086a308b6ef2698a87fec4b9 Mon Sep 17 00:00:00 2001 From: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Date: Tue, 11 Aug 2026 17:17:05 +0900 Subject: [PATCH] [Doc] Fix typos in speculative decoding docs (#51500) Signed-off-by: lkm2835 Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> --- docs/features/speculative_decoding/README.md | 8 ++++---- docs/features/speculative_decoding/draft_model.md | 4 ++-- docs/features/speculative_decoding/mlp.md | 2 +- docs/features/speculative_decoding/n_gram.md | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/features/speculative_decoding/README.md b/docs/features/speculative_decoding/README.md index ceb25dbfd0..1ecc9ac719 100644 --- a/docs/features/speculative_decoding/README.md +++ b/docs/features/speculative_decoding/README.md @@ -1,6 +1,6 @@ # Speculative Decoding -This document shows how to use [Speculative Decoding](https://arxiv.org/pdf/2302.01318) with vLLM to reduce inter-token latency under medium-to-low QPS (query per second), memory-bound workloads. +This document shows how to use [Speculative Decoding](https://arxiv.org/pdf/2302.01318) with vLLM to reduce inter-token latency under medium-to-low QPS (queries per second), memory-bound workloads. To train your own draft models for optimized speculative decoding, see [vllm-project/speculators](speculators.md) for seamless training and integration with vLLM. @@ -86,7 +86,7 @@ only apply to model-based methods such as `draft_model`, `mtp`, `eagle3`, and | `parallel_drafting` | `boolean` | `false` | Enable parallel draft token generation. Only compatible with EAGLE and draft-model methods. | | `rejection_sample_method` | `string` | `strict` | `strict`, `probabilistic`, or `synthetic`. | | `synthetic_acceptance_rate` | `float` | `None` | Average acceptance rate to target when `rejection_sample_method` is `synthetic`. Valid range is `[0, 1]`. | - | `use_heterogeneous_vocab` | `boolean` | `false` | Allow draft and target models with different vocabularies. Builds a token-level intersection at initialisation and constrains draft logits to shared tokens only. Only compatible with `method=draft_model`. Probabilistic draft sampling (`draft_sample_method='probabilistic'`) is not yet supported when this option is enabled. | +| `use_heterogeneous_vocab` | `boolean` | `false` | Allow draft and target models with different vocabularies. Builds a token-level intersection at initialisation and constrains draft logits to shared tokens only. Only compatible with `method=draft_model`. Probabilistic draft sampling (`draft_sample_method='probabilistic'`) is not yet supported when this option is enabled. | !!! note Gemma 4 assistant checkpoints are handled as Gemma 4 MTP speculators, not @@ -200,7 +200,7 @@ speculative decoding, breaking down the guarantees into three key areas: > distribution. [View Test Code](https://github.com/vllm-project/vllm/blob/47b65a550866c7ffbd076ecb74106714838ce7da/tests/samplers/test_rejection_sampler.py#L252) > - **Greedy Sampling Equality**: Confirms that greedy sampling with speculative decoding matches greedy sampling > without it. This verifies that vLLM's speculative decoding framework, when integrated with the vLLM forward pass and the vLLM rejection sampler, - > provides a lossless guarantee. Almost all of the tests in [tests/spec_decode/e2e](../../../tests/v1/spec_decode). + > provides a lossless guarantee. Almost all of the tests in [tests/spec_decode/e2e](../../../tests/v1/spec_decode) > verify this property using [this assertion implementation](https://github.com/vllm-project/vllm/blob/b67ae00cdbbe1a58ffc8ff170f0c8d79044a684a/tests/spec_decode/e2e/conftest.py#L291) 3. **vLLM Logprob Stability** @@ -220,7 +220,7 @@ For mitigation strategies, please refer to the FAQ entry *Can the output of a pr ## Known Feature Incompatibility 1. Pipeline parallelism is not composable with speculative decoding as of `vllm<=0.15.0` -2. Speculative decoding with a draft models is not supported in `vllm<=0.10.0` +2. Speculative decoding with draft models is not supported in `vllm<=0.10.0` ## Resources for vLLM contributors diff --git a/docs/features/speculative_decoding/draft_model.md b/docs/features/speculative_decoding/draft_model.md index 636c797324..5ab4629409 100644 --- a/docs/features/speculative_decoding/draft_model.md +++ b/docs/features/speculative_decoding/draft_model.md @@ -38,7 +38,7 @@ vllm serve Qwen/Qwen3-4B-Thinking-2507 \ --speculative-config '{"model": "Qwen/Qwen3-0.6B", "num_speculative_tokens": 5, "method": "draft_model"}' ``` -The code used to request as completions as a client remains unchanged: +The code used to request completions as a client remains unchanged: ??? code @@ -80,7 +80,7 @@ The code used to request as completions as a client remains unchanged: By default, vLLM requires the draft and target models to share the same vocabulary. Setting `use_heterogeneous_vocab: true` enables the **Token-Level Intersection (TLI)** algorithm, which allows draft models from a different model family with a different tokenizer. - Currently,`use_heterogeneous_vocab` currently requires `draft_sample_method='greedy'` (the default). Probabilistic draft sampling is not yet supported and will be added in a + Currently, `use_heterogeneous_vocab` requires `draft_sample_method='greedy'` (the default). Probabilistic draft sampling is not yet supported and will be added in a future release. ```python diff --git a/docs/features/speculative_decoding/mlp.md b/docs/features/speculative_decoding/mlp.md index 5b26474699..1af678a860 100644 --- a/docs/features/speculative_decoding/mlp.md +++ b/docs/features/speculative_decoding/mlp.md @@ -1,6 +1,6 @@ # MLP Draft Models -The following code configures vLLM to use speculative decoding where proposals are generated by draft models that condition draft predictions on both context vectors and sampled tokens. For more information see [The Hitchhiker's Guide to Speculative Decoding](https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/) and [IBM Research's Technical Report](https://arxiv.org/abs/2404.19124). +The following code configures vLLM to use speculative decoding where proposals are generated by draft models that condition draft predictions on both context vectors and sampled tokens. For more information, see [The Hitchhiker's Guide to Speculative Decoding](https://pytorch.org/blog/hitchhikers-guide-speculative-decoding/) and [IBM Research's Technical Report](https://arxiv.org/abs/2404.19124). ## MLP Drafter Example diff --git a/docs/features/speculative_decoding/n_gram.md b/docs/features/speculative_decoding/n_gram.md index dfb5df6808..7346875ea8 100644 --- a/docs/features/speculative_decoding/n_gram.md +++ b/docs/features/speculative_decoding/n_gram.md @@ -1,7 +1,7 @@ # N-Gram Speculation The following code configures vLLM to use speculative decoding where proposals are generated by -matching n-grams in the prompt. For more information read [this thread.](https://x.com/joao_gante/status/1747322413006643259) +matching n-grams in the prompt. For more information, read [this thread](https://x.com/joao_gante/status/1747322413006643259). ```python from vllm import LLM, SamplingParams