[Model] Add num_cached_tokens for PoolingRequestOutput (#27378)
Signed-off-by: wang.yuqi <noooop@126.com>
This commit is contained in:
@@ -230,6 +230,7 @@ class RequestState:
|
||||
return PoolingRequestOutput(
|
||||
request_id=request_id,
|
||||
outputs=first_output,
|
||||
num_cached_tokens=self.num_cached_tokens,
|
||||
prompt_token_ids=self.prompt_token_ids,
|
||||
finished=finished,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user