4 Commits

Include renames

Author	SHA1	Message	Date
SangBin Cho	603ad84815	[Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309)	2024-04-26 13:02:02 +00:00
SangBin Cho	0ae11f78ab	[Mypy] Part 3 fix typing for nested directories for most of directory (#4161)	2024-04-22 21:32:44 -07:00
3 people	a37d815b83	Make initialization of tokenizer and detokenizer optional (#3748) ... Co-authored-by: Yun Ding <yunding@nvidia.com> Co-authored-by: Roger Wang <ywang@roblox.com>	2024-04-21 22:06:46 +00:00
Cade Daniel	e95cd87959	[Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894)	2024-04-16 13:09:21 -07:00