 Wentao Ye
|
e26736973a
|
[Model Runner V2] Fix prompt logprobs calculation Sizes of tensors must match error (#42778)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
|
2026-05-18 08:27:21 -07:00 |
|
|
PatchyTISandPatchouliTaisa
|
0162596603
|
[Model Runner V2] FP32 gumbel sampling. (#41775)
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
|
2026-05-15 09:20:08 -07:00 |
|
|
Lucas Kabela
|
4e498b5e5c
|
[Bugfix][Performance Improvement] Improve penalties triton kernel performance (#40657)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
|
2026-05-12 05:47:20 +00:00 |
|
|
|
d7af6b34d8
|
[Model Runner V2] Bug fix: logprob dtype int64/int32 issue (#41761)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 21:55:43 +00:00 |
|
|
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
bbee532988
|
[Perf][1/n] Eliminate various GPU<->CPU syncs (#41429)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
2026-05-11 20:36:03 +00:00 |
|
|
Nick Hill
|
989c176c0a
|
[Perf][3/n] Eliminate GPU<->CPU syncs in attention impls (#41434)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-07 19:44:24 -07:00 |
|
|
Giancarlo Delfin
|
e1e4646b06
|
[Model Runner V2] Rebuild attn metadata between draft decode steps (#41162)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-05-05 00:44:55 +00:00 |
|
|
Joachim StudniaandClaude Opus 4.6
|
422dd02598
|
[bugfix] Fix prompt logprobs on request eviction during chunked prefill (#41411)
Signed-off-by: Joachim Studnia <joachim@mistral.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
2026-05-04 11:46:00 -07:00 |
|
|
Wentao YeandNick Hill
|
51295793a2
|
[Model Runner V2] Add logprob_token_ids support (#40559)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-05-01 10:02:03 -07:00 |
|
|
Wentao YeandNick Hill
|
66cc3fa559
|
[Model Runner V2] Multiple prompt logprobs support (#39937)
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
|
2026-04-21 15:49:05 +00:00 |
|
|
Giancarlo Delfin
|
5daf62271d
|
[Model Runner V2] Fuse probabilistic rejection sample kernels (#38496)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-04-07 17:37:37 -07:00 |
|
|
Woosuk Kwon
|
a5e9d511de
|
[MRV2] Use FP64 for Gumbel noise (#37798)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-22 12:28:10 -07:00 |
|
|
Woosuk Kwon
|
96efb91480
|
[Model Runner V2] Fix processed logits in sample() (#37144)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-03-16 00:35:49 -07:00 |
|
|
Nick Hill
|
cd32d6f586
|
[Model Runner V2] Some code simplification (#36929)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-03-13 00:59:23 +00:00 |
|
|
Giancarlo Delfin
|
c77181e534
|
[Model Runner V2] Add probabilistic rejection sampling for spec decoding (#35461)
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
|
2026-03-11 14:04:32 -07:00 |
|
|
Andy Lo
|
0a7165fd71
|
[ModelRunnerV2] Rename sampler functions and variables for clarity (#35459)
Signed-off-by: Andy Lo <andy@mistral.ai>
|
2026-03-02 19:48:56 -08:00 |
|
|
Woosuk Kwon
|
5fcb0cdd68
|
[Model Runner V2] Use FP32 for Gumbel Noise (#34854)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-18 17:07:37 -08:00 |
|
|
Woosuk Kwon
|
d00df624f3
|
[Model Runner V2] Minor refactoring for penalties (#34662)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-16 21:43:00 -08:00 |
|
|
Woosuk Kwon
|
9752da9d9c
|
[Model Runner V2] Minor simplification for BadWordsState (#34669)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-16 21:27:24 -08:00 |
|
|
zhrrrandWoosuk Kwon
|
387a1898d9
|
[Model Runner V2] support bad_words sampling param (#33433)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-16 16:36:06 -08:00 |
|
|
Woosuk Kwon
|
9ca768c740
|
[Model Runner V2] Minor cleanup for Sampler (#34563)
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
|
2026-02-14 18:29:03 -08:00 |
|
|
Nick Hill
|
40218a82ba
|
[ModelRunner V2] Revert token rank comparison difference for now (#34017)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-02-07 11:11:05 +08:00 |
|
|
zhrrr
|
16786da735
|
[Model Runner V2] support apply penalty for spec decode (#33251)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
|
2026-02-06 10:56:48 -08:00 |
|
|
Nick Hill
|
e535d90deb
|
[ModelRunner V2] Misc minor simplifications and optimizations (#33467)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-02-01 22:17:14 +00:00 |
|
|
Nick Hill
|
876a16f4fb
|
[ModelRunner V2] Fix spec decoding + logprobs (#33391)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
|
2026-01-31 03:33:26 +00:00 |
|
|
Woosuk Kwon
|
408195ec59
|
[Model Runner V2] Refactor Prompt Logprobs (#32811)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-21 15:12:20 -08:00 |
|
|
Woosuk Kwon
|
e9c83cdc51
|
[Model Runner V2] Skip kernel launch for penalties & logit_bias (#32634)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-19 22:20:19 -08:00 |
|
|
Woosuk Kwon
|
6c01ffb897
|
[Model Runner V2] Decouple temperature from penalties (#32629)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-19 19:13:24 -08:00 |
|
|
Woosuk Kwon
|
90c0836902
|
[Model Runner V2] Refactor Sampler (#32245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-13 17:58:12 -08:00 |
|
|
Woosuk Kwon
|
dec28688c5
|
[Model Runner V2] Minor refactor for logit_bias (#32209)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-12 13:08:30 -08:00 |
|
|
Woosuk Kwon
|
ca81811bfe
|
[Model Runner V2] Support logit_bias, allowed_token_ids, min_tokens (#32163)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-12 11:31:10 -08:00 |
|
|
Woosuk Kwon
|
025a32f9ed
|
[Model Runner V2] Remove async barrier (#32083)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2026-01-11 20:24:30 -08:00 |
|
|
Woosuk Kwon
|
d471b2aff0
|
[Model Runner V2] Support num NaNs in logits (#30187)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-12-09 10:00:49 -08:00 |
|
|
Woosuk Kwon
|
9e6562a3f6
|
[Model Runner V2] Fix Triton warning on tl.where (#30355)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-12-09 09:59:54 -08:00 |
|
|
Woosuk Kwon
|
a238cbd89d
|
[Model Runner V2] Support min-p sampling (#30171)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-12-05 21:42:47 -08:00 |
|
|
Woosuk Kwon
|
ec38a7368d
|
[Model Runner V2] Use packed mask for prompt bin counts (#29756)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-30 14:15:42 -08:00 |
|
|
Woosuk Kwon
|
f223ed4181
|
[Model Runner V2] Fuse penalties and temperature into single kernel (#29720)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-29 02:29:16 -08:00 |
|
|
Woosuk Kwon
|
6afc0ffaf6
|
[Model Runner V2] Add sample/ directory and reorganize files (#29719)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2025-11-29 00:41:01 -08:00 |
|