[Bugfix][CPU] Fix fused Gumbel-max wrapping-window bias (#60221)

Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Bob <bob@ibm.com>
This commit is contained in:
Rehan Khan
2026-10-08 15:15:53 +08:00
committed by GitHub
co-authored by Bob
parent e58f8dde7f
commit fd0216d3b5
6 changed files with 236 additions and 80 deletions
+4 -4
View File
@@ -319,11 +319,11 @@ class TopKTopPSampler(nn.Module):
k: torch.Tensor | None,
p: torch.Tensor | None,
) -> tuple[torch.Tensor, torch.Tensor | None]:
"""Fused Gumbel-max sampling for CPU.
"""CPU Gumbel-max sampling with per-token SplitMix64 noise.
Uses a precomputed Gumbel table + single-pass argmax over logits,
skipping softmax and intermediate allocations entirely.
Falls back to the native path when fp64 Gumbel noise is requested.
Independent (seed, token) streams avoid the wrapping-window bias
(#59786). Falls back to native sampling when fp64 Gumbel noise
is requested.
"""
logits = apply_top_k_top_p(logits, k, p)
logits_to_return = None