[Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends (#61574)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude
Robert Shaw
parent
92dc3dd393
commit
4ac1f8d2ac
@@ -28,6 +28,9 @@ class StructuredOutputRequest:
|
||||
# Cached per request; do not share reasoning parsers across requests because
|
||||
# their behavior can depend on reasoning_parser_kwargs.
|
||||
reasoner: "ReasoningParser | None" = None
|
||||
# Implicit reasoning-end tokens the grammar rejects as the first content
|
||||
# token; masked while the request reasons. Computed on first use.
|
||||
masked_reasoning_end_ids: tuple[int, ...] | None = None
|
||||
|
||||
@staticmethod
|
||||
def from_sampling_params(
|
||||
|
||||
Reference in New Issue
Block a user