iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

[Perf] Optimize hidden state extraction logic (#37374)

Browse Source 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit is contained in:
Benjamin Chislett
2026-05-22 18:23:08 -04:00
committed by GitHub
co-authored by gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
parent f743254143
commit 4e2eba28be
9 changed files with 772 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/features/speculative_decoding/README.md
+1
View File
@@ -15,6 +15,7 @@ vLLM supports a variety of methods of speculative decoding. Model-based methods
- [Multi-Layer Perceptron](mlp.md)
- [N-Gram](n_gram.md)
- [Suffix Decoding](suffix.md)
- [Hidden State Extraction](extract_hidden_states.md)
- [Custom Proposer Backend (Experimental)](#custom-proposer-backend-experimental)
## Method Selection at a Glance