Signed-off-by: Karthik Kothuri <karthikkothuri2009@gmail.com> Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
This commit is contained in:
co-authored by
Claude
wang.yuqi
parent
56a357ed33
commit
107a03ba63
+5
-11
@@ -1709,24 +1709,18 @@ class ModelConfig:
|
||||
such as the lm_head in a generation model,
|
||||
or the score or classifier in a classification model.
|
||||
|
||||
`head_dtype` currently only supports pooling models.
|
||||
|
||||
- The pooling model defaults to using fp32 head, you can use
|
||||
- Pooling models default to an fp32 head; use
|
||||
--hf-overrides '{"head_dtype": "model"}' to disable it.
|
||||
- Generation models default to the model dtype; set
|
||||
--hf-overrides '{"head_dtype": "float32"}' to run the lm_head in
|
||||
fp32, which is required for RL training-inference consistency
|
||||
(the trainer computes logits in fp32).
|
||||
"""
|
||||
|
||||
head_dtype = _get_head_dtype(
|
||||
config=self.hf_config, dtype=self.dtype, runner_type=self.runner_type
|
||||
)
|
||||
|
||||
if self.runner_type != "pooling" and head_dtype != self.dtype:
|
||||
logger.warning_once(
|
||||
"`head_dtype` currently only supports pooling models, "
|
||||
"fallback to model dtype [%s].",
|
||||
self.dtype,
|
||||
)
|
||||
return self.dtype
|
||||
|
||||
if head_dtype not in current_platform.supported_dtypes:
|
||||
logger.warning_once(
|
||||
"The current platform does not support [%s] head dtype, "
|
||||
|
||||
Reference in New Issue
Block a user