Cyrus Leung 51e971d39e [Bugfix] Support eos_token_id from config.json (#5954)

2024-06-29 11:19:02 +00:00

..

configs

[BugFix] Fix MLPSpeculator handling of num_speculative_tokens (#5876)

2024-06-27 10:59:33 -07:00

tokenizer_group

[Core] Add fault tolerance for RayTokenizerGroupPool (#5748)

2024-06-25 10:15:10 -07:00

tokenizers

[Mypy] Part 3 fix typing for nested directories for most of directory (#4161)

2024-04-22 21:32:44 -07:00

__init__.py

[Tokenizer] Add an option to specify tokenizer (#284)

2023-06-28 09:46:58 -07:00

config.py

[Bugfix] Support eos_token_id from config.json (#5954)

2024-06-29 11:19:02 +00:00

detokenizer.py

[mypy] Enable type checking for test directory (#5017)

2024-06-15 04:45:31 +00:00

image_processor.py

[Core] Registry for processing model inputs (#5214)

2024-06-28 12:09:56 +00:00

tokenizer.py

[Misc] centralize all usage of environment variables (#4548)

2024-05-02 11:13:25 -07:00