[Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359)

Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
vx120
2026-10-01 13:51:39 +08:00
committed by GitHub
co-authored by Cuiping Cheng aoshen02 Claude Opus 5.5
parent 37cecc8141
commit 70be6aae19
13 changed files with 583 additions and 68 deletions
+4 -1
View File
@@ -63,9 +63,12 @@ The mask is also available via the `/inference/v1/generate` HTTP endpoint:
| `top_k > 0` | Bounds mask size; pure top-p can produce vocab-sized masks |
| Model Runner V2 | Required by the async D2H copy pipeline |
Speculative decoding is supported; each emitted token gets the support of the
target distribution at its position.
The engine rejects unsupported combinations at startup or request time:
- Speculative decoding
- Speculative decoding with adaptive verification
- Diffusion models
- Custom logits processors (engine-level `--logits-processors`)