[Feature][Spec Decode] Support sampling mask replay for MRV2 MTP (#59359)
Signed-off-by: Cuiping Cheng <chengcuipingswu@163.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Cuiping Cheng <chengcuipingswu@163.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Cuiping Cheng
aoshen02
Claude Opus 5.5
parent
37cecc8141
commit
70be6aae19
@@ -441,7 +441,11 @@ class RequestState:
|
||||
sampling_mask = None
|
||||
if finished and self.sampling_mask_chunks:
|
||||
sampling_mask = SamplingMask(
|
||||
[chunk.token_ids.tolist() for chunk in self.sampling_mask_chunks]
|
||||
[
|
||||
position
|
||||
for chunk in self.sampling_mask_chunks
|
||||
for position in chunk.to_nested_list()
|
||||
]
|
||||
)
|
||||
|
||||
# Concatenate routed experts on finish
|
||||
|
||||
Reference in New Issue
Block a user