[Hy4] support Hy4-preview model (#54160)

Signed-off-by: chengvjiang <chengvjiang@tencent.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
This commit is contained in:
Cheng Jiang
2026-08-29 12:07:28 +08:00
committed by GitHub
co-authored by chengvjiang Jee Jee Li Bugen Zhao russellfeng Jee Jee Li
parent 6c18a54648
commit b2f685834a
47 changed files with 6260 additions and 217 deletions
+5
View File
@@ -552,6 +552,11 @@ def dummy_hf_overrides(
text_config.update(update_dict)
# Keep per-layer metadata consistent with the reduced layer count.
# HY V4 uses this to decide which sparse-attention indexers are shared.
if indexer_types := getattr(text_config, "indexer_types", None):
text_config.update({"indexer_types": indexer_types[:num_hidden_layers]})
# Update n_layers and moe configs for Moondream3 model
if model_arch in ("Moondream3ForCausalLM", "HfMoondream"):
text_config.update(