[Model] Add num_cached_tokens for PoolingRequestOutput (#27378)

Signed-off-by: wang.yuqi <noooop@126.com>
This commit is contained in:
wang.yuqi
2025-10-23 14:03:42 +08:00
committed by GitHub
parent 6644796bf4
commit 3729ed00ba
7 changed files with 75 additions and 5 deletions
+1
View File
@@ -230,6 +230,7 @@ class RequestState:
return PoolingRequestOutput(
request_id=request_id,
outputs=first_output,
num_cached_tokens=self.num_cached_tokens,
prompt_token_ids=self.prompt_token_ids,
finished=finished,
)