[Frontend] Cohere chat v2 api support (#47189)
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
This commit is contained in:
co-authored by
Cyrus Leung
parent
6c91de3689
commit
9c110fa522
@@ -93,6 +93,7 @@ TokenizerMode = Literal[
|
||||
"deepseek_v4",
|
||||
"inkling",
|
||||
"kimi_k3",
|
||||
"cohere",
|
||||
]
|
||||
ModelDType = Literal["auto", "half", "float16", "bfloat16", "float", "float32"]
|
||||
LogprobsMode = Literal[
|
||||
@@ -151,6 +152,9 @@ class ModelConfig:
|
||||
- "deepseek_v4" will always use the tokenizer from `deepseek_v4`.
|
||||
- "kimi_k3" will always use the "hf" tokenizer but render chat prompts
|
||||
with Kimi K3's Python XTML encoding instead of a Jinja template.
|
||||
- "cohere" uses the standard HF tokenizer but renders the chat template
|
||||
via the `cohere_melody` library (cmd3 / cmd4 templates) instead of
|
||||
Jinja, and surfaces grounded-citation metadata on responses.
|
||||
- Other custom values can be supported via plugins.
|
||||
|
||||
To swap the Rust BPE backend that powers HF fast tokenizers for the
|
||||
|
||||
Reference in New Issue
Block a user