[Bugfix][Sampler] Carry the boundary tie count across blocks in the split top-p mask kernel (#62046)

Signed-off-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com>
Co-authored-by: Bowen Wang <bowen@inferact.ai>
This commit is contained in:
Akshat Anand
2026-10-17 02:26:57 +00:00
committed by GitHub
co-authored by cipheraxat Bowen Wang
parent ce8853d27a
commit ce49c47cbb
2 changed files with 31 additions and 0 deletions
+1
View File
@@ -1513,6 +1513,7 @@ def _topp_sb_mask_kernel(
cum = tl.cumsum(dmask.to(tl.int32)).to(tl.float32) + num_preceding
keep_dup = dmask & (cum <= numkeep)
keep = keep & (~dmask | keep_dup)
num_preceding += tl.sum(dmask.to(tl.float32))
tl.store(ROW + i + offs, tl.where(keep, x, MASK_VALUE), mask=mask_n)