[Bugfix] Fix thinking_token_budget not enforced after natural </think> re-entry (#45984)
Signed-off-by: Ashwin Giridharan <girida@amazon.com>
This commit is contained in:
@@ -113,6 +113,7 @@ class InputBatch:
|
||||
max_num_reqs,
|
||||
num_spec_tokens,
|
||||
device,
|
||||
PIN_MEMORY,
|
||||
)
|
||||
self.thinking_token_budget_reqs: set[str] = set()
|
||||
self.is_pooling_model = is_pooling_model
|
||||
|
||||
Reference in New Issue
Block a user