Julien Denize 6d8d0a24c0 Add think chunk (#21333) ...

Signed-off-by: Julien Denize <julien.denize@mistral.ai>

2025-07-23 21:51:32 -07:00

..

chat_templates

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

configs

[Bugfix] Fix nightly transformers CI failure (#21427)

2025-07-23 00:01:01 -07:00

processors

[CI] change spell checker from codespell to typos (#18711)

2025-06-11 19:57:10 -07:00

tokenizers

Add think chunk (#21333)

2025-07-23 21:51:32 -07:00

__init__.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

config.py

[Bugfix] Fix nightly transformers CI failure (#21427)

2025-07-23 00:01:01 -07:00

detokenizer_utils.py

[Bugfix]: Fix messy code when using logprobs (#20910)

2025-07-14 11:06:45 +00:00

detokenizer.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

processor.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

s3_utils.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

tokenizer_base.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

tokenizer_group.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

tokenizer.py

[Bugfix] The special_tokens in tokenizer should also be controlled by do_lower_case in encoder_config. (#20750)

2025-07-18 09:10:47 +00:00

utils.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00