[Bugfix][Spec Decode][Structured Output] DSpark: fix the grammar bitmask mapping when the draft budget is zero (#52436)
Signed-off-by: oops-oom <73481342@qq.com> Co-authored-by: oops-oom <73481342@qq.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
oops-oom
Claude Opus 4.8
parent
84530eb235
commit
4d2a68d64d
@@ -425,6 +425,7 @@ class GPUModelRunner(LoRAModelRunnerMixin):
|
||||
vocab_size=self.vocab_size,
|
||||
device=self.device,
|
||||
mask_stride=self.decode_query_len,
|
||||
num_bonus_tokens=self.model_state.num_new_sampled_tokens_per_step,
|
||||
)
|
||||
|
||||
if self.is_pooling_model and self.is_last_pp_rank:
|
||||
|
||||
Reference in New Issue
Block a user