[Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359)
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Cuiping Cheng
aoshen02
Claude Opus 5.5
parent
37cecc8141
commit
70be6aae19
@@ -63,9 +63,12 @@ The mask is also available via the `/inference/v1/generate` HTTP endpoint:
|
||||
| `top_k > 0` | Bounds mask size; pure top-p can produce vocab-sized masks |
|
||||
| Model Runner V2 | Required by the async D2H copy pipeline |
|
||||
|
||||
Speculative decoding is supported; each emitted token gets the support of the
|
||||
target distribution at its position.
|
||||
|
||||
The engine rejects unsupported combinations at startup or request time:
|
||||
|
||||
- Speculative decoding
|
||||
- Speculative decoding with adaptive verification
|
||||
- Diffusion models
|
||||
- Custom logits processors (engine-level `--logits-processors`)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user