[Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359)

Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
vx120
2026-10-01 13:51:39 +08:00
committed by GitHub
co-authored by Cuiping Cheng aoshen02 Claude Opus 5.5
parent 37cecc8141
commit 70be6aae19
13 changed files with 583 additions and 68 deletions
+4 -1
View File
@@ -203,7 +203,10 @@ class Sampler:
# Size by the validated top_k batch max; wider supports use the bitmask.
max_num_kept = int(np.max(self.sampling_states.top_k.np[idx_mapping_np]))
sampling_mask_tensors = SamplingMaskTensors.from_logits(
processed_logits, num_sampled, max_num_kept
processed_logits,
input_batch.cu_num_logits,
num_sampled,
max_num_kept,
)
# These are GPU tensors.