3 people 3facf822b7 [Performance] Load HF tokenizers and processors from a local snapshot (#60825) ...

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>

2026-10-12 13:45:54 +00:00

..

fixtures

[Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316)

2026-09-26 02:18:58 +00:00

__init__.py

[Misc] Refactor tokenizer interface (#29693)

2025-11-29 04:02:21 -08:00

test_basic.py

[Performance] Load HF tokenizers and processors from a local snapshot (#60825)

2026-10-12 13:45:54 +00:00

test_deepseek_v4.py

[Misc] mv chat_utils.py to renderers. (#60522)

2026-10-10 06:50:34 +00:00

test_deepseek_v41.py

[Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasoning effort mappings (#58316)

2026-09-26 02:18:58 +00:00

test_detokenize.py

[Perf] Reduce fallback decode CPU by 69% at 32K (#60497)

2026-10-08 21:42:56 +00:00

test_hf.py

[Bugfix][Tokenizer] Fix off-by-one max_token_id from vocab_size (#59491)

2026-10-01 05:43:30 +00:00

test_mistral.py

[Misc] mv chat_utils.py to renderers. (#60522)

2026-10-10 06:50:34 +00:00

test_registry.py

[Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623)

2026-10-09 09:29:03 +00:00