 Cade Daniel
|
baa240252e
|
[Core] Fix edge case in chunked prefill + block manager v2 (#7380)
|
2024-08-09 23:48:49 +00:00 |
|
|
Cade Daniel
|
82a1b1a82b
|
[Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963)
|
2024-08-05 08:46:44 +00:00 |
|
|
Cade Daniel
|
c32ab8be1a
|
[Speculative decoding] Add serving benchmark for llama3 70b + speculative decoding (#6964)
|
2024-07-31 00:53:21 +00:00 |
|
|
Cade Daniel
|
fb4f530bf5
|
[CI] [nightly benchmark] Do not re-download sharegpt dataset if exists (#6706)
|
2024-07-30 16:28:49 -07:00 |
|
|
Cade Daniel
|
79319cedfa
|
[Nightly benchmarking suite] Remove pkill python from run benchmark suite (#6965)
|
2024-07-30 16:28:05 -07:00 |
|
|
Cade Daniel
|
89ec06c33b
|
[Docs] [Spec decode] Fix docs error in code example (#5427)
|
2024-06-11 10:31:56 -07:00 |
|
|
Cade Daniel
|
4c2ffb28ff
|
[Speculative decoding] Initial spec decode docs (#5400)
|
2024-06-11 10:15:40 -07:00 |
|
|
Cade Daniel
|
8b9241be3a
|
[Speculative decoding] [Bugfix] Fix overallocation in ngram + spec logprobs (#4672)
|
2024-05-08 23:24:46 +00:00 |
|
|
Cade Daniel
|
19cb4716ee
|
[CI] Add retry for agent lost (#4633)
|
2024-05-06 23:18:57 +00:00 |
|
|
Cade Daniel
|
ab50275111
|
[Speculative decoding] Support target-model logprobs (#4378)
|
2024-05-03 15:52:01 -07:00 |
|
|
Cade Daniel
|
62b8aebc6f
|
[Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951)
|
2024-04-23 08:02:36 +00:00 |
|
|
Cade Daniel
|
11d652bd4f
|
[CI] Move CPU/AMD tests to after wait (#4123)
|
2024-04-16 22:53:26 -07:00 |
|
|
Cade Daniel
|
d150e4f89f
|
[Misc] [CI] Fix CI failure caught after merge (#4126)
|
2024-04-16 17:56:01 -07:00 |
|
|
Cade Daniel
|
e95cd87959
|
[Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894)
|
2024-04-16 13:09:21 -07:00 |
|
|
Cade Daniel
|
e7c7067b45
|
[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" (#3837)
|
2024-04-09 11:44:15 -07:00 |
|
|
Cade Daniel
|
e0dd4d3589
|
[Misc] Fix linter issues in examples/fp8/quantizer/quantize.py (#3864)
|
2024-04-04 21:57:33 -07:00 |
|
|
Cade Daniel
|
e5043a3e75
|
[Misc] Add pytest marker to opt-out of global test cleanup (#3863)
|
2024-04-04 21:54:16 -07:00 |
|
|
Cade DanielandLily Liu
|
5757d90e26
|
[Speculative decoding] Adding configuration object for speculative decoding (#3706)
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
|
2024-04-03 00:40:57 +00:00 |
|
|
Cade Daniel
|
eb69d68804
|
[Misc] [CI/Build] Speed up block manager CPU-only unit tests ~10x by opting-out of GPU cleanup (#3783)
|
2024-04-02 00:49:51 +00:00 |
|
|
Cade Daniel
|
93deb0b38f
|
[Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250)
|
2024-04-01 22:55:24 +00:00 |
|
|
Cade Daniel
|
14ccd94c89
|
[Core][Bugfix]Refactor block manager for better testability (#3492)
|
2024-03-27 23:59:28 -07:00 |
|
|
Cade Daniel
|
482b0adf1b
|
[Testing] Add test_config.py to CI (#3437)
|
2024-03-18 12:48:45 -07:00 |
|
|
Cade Daniel
|
8437bae6ef
|
[Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103)
|
2024-03-08 23:32:46 -08:00 |
|
|
Cade Daniel
|
a33ce60c66
|
[Testing] Fix core tests (#3224)
|
2024-03-06 01:04:23 -08:00 |
|
|
Cade Daniel
|
18bfcdd05c
|
[Speculative decoding 2/9] Multi-step worker for draft model (#2424)
|
2024-01-21 16:31:47 -08:00 |
|
|
Cade Daniel
|
79d64c4954
|
[Speculative decoding 1/9] Optimized rejection sampler (#2336)
|
2024-01-09 15:38:41 -08:00 |
|
|
Cade Daniel
|
e575df33b1
|
[Small] Formatter only checks lints in changed files (#1528)
|
2023-10-31 15:39:38 -07:00 |
|