leiwen83andLei Wen b38e42fbca [Speculative decoding] Add ngram prompt lookup decoding (#4237) ...

Co-authored-by: Lei Wen <wenlei03@qiyi.com>

2024-05-01 11:13:03 -07:00

..

output_processor

[Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309)

2024-04-26 13:02:02 +00:00

__init__.py

Change the name to vLLM (#150)

2023-06-17 03:07:40 -07:00

arg_utils.py

[Speculative decoding] Add ngram prompt lookup decoding (#4237)

2024-05-01 11:13:03 -07:00

async_llm_engine.py

[BugFix] fix num_lookahead_slots missing in async executor (#4165)

2024-04-30 10:12:59 -07:00

llm_engine.py

[Misc][Typo] type annotation fix (#4495)

2024-04-30 20:21:39 -07:00

metrics.py

[Bugfix] Fix 307 Redirect for /metrics (#4523)

2024-05-01 09:14:13 -07:00