[Feature] limit thinking tokens (hard limit) (#20859)

Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
Signed-off-by: Sungjae Lee <sung-jae.lee@navercorp.com>
Signed-off-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Sungjae Lee
2026-03-24 09:53:07 -07:00
committed by GitHub
co-authored by Chauncey Harry Mellor
parent 8de5261e69
commit 4731884796
13 changed files with 702 additions and 12 deletions
+10
View File
@@ -99,6 +99,16 @@ class InputProcessor:
self.structured_outputs_config,
self.tokenizer,
)
if (
params.thinking_token_budget is not None
and self.vllm_config.reasoning_config is None
):
raise ValueError(
"thinking_token_budget is set but reasoning_config is "
"not configured. Please set --reasoning-config to use "
"thinking_token_budget."
)
elif isinstance(params, PoolingParams):
supported_pooling_tasks = [
task for task in supported_tasks if task in POOLING_TASKS