[Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046)
Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com> Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com> Co-authored-by: Bowen Wang <bowen@inferact.ai>
This commit is contained in:
co-authored by
cipheraxat
Bowen Wang
parent
ce8853d27a
commit
ce49c47cbb
@@ -1513,6 +1513,7 @@ def _topp_sb_mask_kernel(
|
||||
cum = tl.cumsum(dmask.to(tl.int32)).to(tl.float32) + num_preceding
|
||||
keep_dup = dmask & (cum <= numkeep)
|
||||
keep = keep & (~dmask | keep_dup)
|
||||
num_preceding += tl.sum(dmask.to(tl.float32))
|
||||
tl.store(ROW + i + offs, tl.where(keep, x, MASK_VALUE), mask=mask_n)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user