[Frontend] Cohere chat v2 api support (#47189)

Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
This commit is contained in:
andrewbcohere
2026-07-31 22:53:54 -07:00
committed by GitHub
co-authored by Cyrus Leung
parent 6c91de3689
commit 9c110fa522
31 changed files with 8770 additions and 20 deletions
+4
View File
@@ -93,6 +93,7 @@ TokenizerMode = Literal[
"deepseek_v4",
"inkling",
"kimi_k3",
"cohere",
]
ModelDType = Literal["auto", "half", "float16", "bfloat16", "float", "float32"]
LogprobsMode = Literal[
@@ -151,6 +152,9 @@ class ModelConfig:
- "deepseek_v4" will always use the tokenizer from `deepseek_v4`.
- "kimi_k3" will always use the "hf" tokenizer but render chat prompts
with Kimi K3's Python XTML encoding instead of a Jinja template.
- "cohere" uses the standard HF tokenizer but renders the chat template
via the `cohere_melody` library (cmd3 / cmd4 templates) instead of
Jinja, and surfaces grounded-citation metadata on responses.
- Other custom values can be supported via plugins.
To swap the Rust BPE backend that powers HF fast tokenizers for the