[Bugfix] Reject negative values for max_logprobs and long_prefill_token_threshold (#44002)
Signed-off-by: jwzheng96 <jianweizheng@pku.edu.cn> Signed-off-by: JianweiZheng <32029023+jwzheng96@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude
Wentao Ye
Harry Mellor
parent
ea9ddf59fc
commit
1907d3854a
@@ -79,7 +79,7 @@ class SchedulerConfig:
|
||||
|
||||
long_prefill_token_threshold: int = Field(default=0, ge=0)
|
||||
"""For chunked prefill, a request is considered long if the prompt is
|
||||
longer than this number of tokens."""
|
||||
longer than this number of tokens. 0 disables the cap (default)."""
|
||||
|
||||
enable_chunked_prefill: bool = True
|
||||
"""If True, prefill requests can be chunked based
|
||||
|
||||
Reference in New Issue
Block a user