[Bugfix][Model Runner V2][Spec Decode] Fix off-by-one in bad_words draft-prefix matching (#52311)
Signed-off-by: jyan <r_02213@sjtu.edu.cn> Co-authored-by: jyan <r_02213@sjtu.edu.cn> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
jyan
Claude Fable 5
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
70aaec832b
commit
1b079c40ff
@@ -153,8 +153,10 @@ def _bad_words_kernel(
|
||||
|
||||
from_spec_input = actual_pos >= output_len
|
||||
if from_spec_input:
|
||||
# input_ids at local position 0 is the last committed token;
|
||||
# draft tokens start at local position 1.
|
||||
spec_offset = actual_pos - output_len
|
||||
actual = tl.load(input_ids_ptr + cur_req_first_pos + spec_offset)
|
||||
actual = tl.load(input_ids_ptr + cur_req_first_pos + spec_offset + 1)
|
||||
else:
|
||||
actual = tl.load(output_base + actual_pos)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user