 Giancarlo DelfinandNick Hill
|
27930df9c2
|
[Model Runner V2] Fix -1 placeholder draft token ids in rejection sam… (#50939)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-08-06 16:32:00 -07:00 |
|
|
Giancarlo Delfin
|
81bc196913
|
[Model Runner V2] Cache draft logits in model's LM head dtype (#50910)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-08-05 19:12:46 -07:00 |
|
|
gabriel-peracioandKimi K3
|
47a4e410ba
|
[Bugfix][Spec Decode] Fix NaN handling in rejection sampler tl.argmax (#50183)
Signed-off-by: Gabriel Peracio <gabriel@sigsonic.com>
Co-authored-by: Kimi K3 <noreply@moonshot.ai>
|
2026-08-05 18:01:09 -07:00 |
|
|
Michael GoinandNick Hill
|
ac36a7a1e7
|
[MRV2][Spec Decode] Avoid rejection sampler OOM by chunking (#48630)
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-07-23 18:13:13 +08:00 |
|
|
Giancarlo Delfin
|
db808b3961
|
[Model Runner V2][Spec Decode] Implement block verification for rejection sampling (#46781)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-06-30 08:07:24 -07:00 |
|
|
Nick Hill
|
7ee4d22009
|
[Spec Decode] Reject placeholder (-1) draft tokens in rejection sampler (#46533)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-06-24 03:32:32 +00:00 |
|
|
Giancarlo Delfin
|
fe5b4e0fe7
|
[Model Runner V2] Apply synthetic mode to probabilistic rejection sampler (#41035)
|
2026-05-12 13:37:03 -07:00 |
|