[Bugfix][Rust Frontend] Fix missing added tokens in hf/fastokens tokenizer (#44683)
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>
This commit is contained in:
@@ -225,7 +225,7 @@ impl MultimodalModelInfo {
|
||||
///
|
||||
/// The HF renderer uses this token while flattening image content in string
|
||||
/// content format.
|
||||
pub(crate) fn placeholder_token(&self) -> &str {
|
||||
pub fn placeholder_token(&self) -> &str {
|
||||
&self.spec.placeholder_token
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user