[Frontend] Report stop_reason on /inference/v1/generate and pass it through derender (#60251)
Signed-off-by: Hyeonki Hong <hhk7734@gmail.com> Signed-off-by: Shimi Bandiel <shimib@google.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Hyeonki Hong <hhk7734@gmail.com> Co-authored-by: Mimir <mimir@nousresearch.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>
This commit is contained in:
co-authored by
Hyeonki Hong
Mimir
Claude
Bugen Zhao
parent
32b05addc1
commit
8d7b93fec2
@@ -206,6 +206,7 @@ class OnlineDerenderer:
|
||||
message=message,
|
||||
logprobs=resolved_logprobs,
|
||||
finish_reason=choice.finish_reason,
|
||||
stop_reason=choice.stop_reason,
|
||||
token_ids=choice.token_ids if return_token_ids else None,
|
||||
)
|
||||
)
|
||||
@@ -484,6 +485,7 @@ class OnlineDerenderer:
|
||||
delta=delta,
|
||||
logprobs=resolved_logprobs,
|
||||
finish_reason=choice.finish_reason,
|
||||
stop_reason=choice.stop_reason,
|
||||
token_ids=choice.token_ids if return_token_ids else None,
|
||||
)
|
||||
)
|
||||
@@ -672,6 +674,7 @@ class OnlineDerenderer:
|
||||
index=choice.index,
|
||||
delta=delta_message,
|
||||
finish_reason=finish_reason,
|
||||
stop_reason=choice.stop_reason,
|
||||
token_ids=choice.token_ids if return_token_ids else None,
|
||||
)
|
||||
stream_choices.append(
|
||||
@@ -777,6 +780,7 @@ class OnlineDerenderer:
|
||||
index=index,
|
||||
text=decoded_text,
|
||||
finish_reason=choice.finish_reason,
|
||||
stop_reason=choice.stop_reason,
|
||||
logprobs=completion_logprobs,
|
||||
)
|
||||
)
|
||||
@@ -878,6 +882,7 @@ class OnlineDerenderer:
|
||||
text=new_text,
|
||||
logprobs=completion_logprobs,
|
||||
finish_reason=choice.finish_reason,
|
||||
stop_reason=choice.stop_reason,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user