[Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends (#61574)

Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
This commit is contained in:
Maroon Ayoub
2026-10-16 12:53:41 +08:00
committed by GitHub
co-authored by Claude Robert Shaw
parent 92dc3dd393
commit 4ac1f8d2ac
7 changed files with 198 additions and 20 deletions
+3
View File
@@ -28,6 +28,9 @@ class StructuredOutputRequest:
# Cached per request; do not share reasoning parsers across requests because
# their behavior can depend on reasoning_parser_kwargs.
reasoner: "ReasoningParser | None" = None
# Implicit reasoning-end tokens the grammar rejects as the first content
# token; masked while the request reasons. Computed on first use.
masked_reasoning_end_ids: tuple[int, ...] | None = None
@staticmethod
def from_sampling_params(