[Feature] Mask Replay (#49577)
Signed-off-by: vx120 <893600387@qq.com> Signed-off-by: vx120 <57470515+vx120@users.noreply.github.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: OpenAI Codex <codex@openai.com>
This commit is contained in:
co-authored by
aoshen02
Codex
Claude Opus 4.6
OpenAI Codex
parent
10bcad2523
commit
50ba4bc6b2
@@ -108,6 +108,17 @@ class InputProcessor:
|
||||
self.tokenizer,
|
||||
)
|
||||
|
||||
if self.model_config.return_sampling_mask:
|
||||
if params.temperature <= 0:
|
||||
raise ValueError(
|
||||
"sampling distribution replay requires temperature > 0"
|
||||
)
|
||||
if params.top_k <= 0:
|
||||
raise ValueError(
|
||||
"sampling distribution replay requires top_k > 0 to "
|
||||
"bound sampling mask size, reduce transfer overhead, "
|
||||
"and avoid potential OOMs"
|
||||
)
|
||||
if params.thinking_token_budget is not None and (
|
||||
self.vllm_config.reasoning_config is None
|
||||
or not self.vllm_config.reasoning_config.enabled
|
||||
|
||||
Reference in New Issue
Block a user