[Bugfix] Fix thinking_token_budget not enforced after natural </think> re-entry (#45984)

Signed-off-by: Ashwin Giridharan <girida@amazon.com>
This commit is contained in:
Ashwin Giridharan
2026-07-10 22:47:51 +00:00
committed by GitHub
parent 26ff616bbf
commit ed908cf0a0
3 changed files with 453 additions and 47 deletions
+1
View File
@@ -113,6 +113,7 @@ class InputBatch:
max_num_reqs,
num_spec_tokens,
device,
PIN_MEMORY,
)
self.thinking_token_budget_reqs: set[str] = set()
self.is_pooling_model = is_pooling_model