148 Commits
Author SHA1 Message Date
4ac1f8d2ac [Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends (#61574)
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
2026-10-16 12:53:41 +08:00
818f70c295 [Bugfix] Mask special added tokens in xgrammar JSON, regex and EBNF grammars (#60749)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-13 14:38:23 +03:00
samayp01andArtem Perevedentsev 54db759eea [Bugfix][Structured Output] Reject structural_tag and unhandled specs in backend validation (#58862)
Signed-off-by: Samay <samay.piduru@gmail.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-13 13:27:34 +03:00
DimensionSTPandArtem Perevedentsev 314ab8cffd [Bugfix][Structured Output] Drain compilation tasks before releasing the backend (#61047)
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-12 16:25:36 +03:00
Kai HoangandFlora Feng e0cb3efce0 [Bugfix][Structured Output] Decode choice specs as JSON in outlines and LMFE (#60748)
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-10-09 20:52:08 +00:00
Willow LopezandArtem Perevedentsev ad8a6065f4 [Bugfix] Validate guidance tokenizer compatibility before EngineCore (#52298)
Signed-off-by: Oxygen56 <jiangth99@163.com>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-09 21:59:02 +03:00
Shreyansh JainandArtem Perevedentsev 28d76f3bad [Bugfix][Structured Output] Reject unsupported regex inside groups (#60359)
Signed-off-by: Shreyansh Jain <shreyanshjain646@gmail.com>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-09 14:16:13 +03:00
f43e4c39e3 [Structured Output] Allow propertyNames with additionalProperties on xgrammar (#57594)
Signed-off-by: errmakov <ide404@gmail.com>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-09 10:49:47 +03:00
c23ca06b6d [Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar (#59627)
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com>
2026-10-08 19:38:33 +00:00
73c742b09c [Bugfix] Make disable_any_whitespace actually disable whitespace on xgrammar (#58067)
Signed-off-by: Stu Cao <stu@primeintellect.ai>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-08 14:41:14 +03:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 02:30:52 -07:00
Artem PerevedentsevandShashank Varma 049507aa76 [Bugfix][Structured Output] Flag multi-branch allOf as unsupported for xgrammar (#59061)
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Shashank Varma <324153016+shashankvarma499@users.noreply.github.com>
2026-10-06 17:01:04 +04:00
Mai Hoàng ĐứcandFlora Feng 5cd8a2626b [Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls (#60036)
Signed-off-by: Mazukiri <maihoangduc04@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
2026-10-06 00:06:59 +00:00
60932a6401 [Bugfix][Structured Output] Preserve literal values in Guidance disable_additional_properties (#58709)
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com>
Co-authored-by: Antigravity <noreply@google.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-10-05 18:12:40 +03:00
Shinichi Hemmi 17cc3cd80e [Model] Engine based plamo3 parser (#56575)
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp>
2026-09-29 05:17:42 +00:00
Bugen ZhaoClaude Opus 5.5mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1f0bc49ee6 [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
2026-09-25 04:35:10 +00:00
SIDDARTHA REDDY a811738a60 [Bugfix] Accept EOS after grammar finish in outlines backend; reject json_object at validation (#57743)
Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com>
2026-09-24 17:17:10 -07:00
Nick Hill 36f94d5fe8 [Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts (#58612) 2026-09-24 21:55:56 +00:00
9815732c16 [BugFix][Core] Make the structured-output grammar poll non-blocking (#55931)
Signed-off-by: ubwzwd <ubwzwd@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
2026-09-22 14:39:47 +03:00
3c84ad13ed [Bugfix] Fix xgrammar feature gate bypass when JSON Schema type is a list (#48416)
Signed-off-by: john <6712432+CaiJohn@users.noreply.github.com>
Co-authored-by: john <6712432+CaiJohn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-09-21 09:59:20 +03:00
CaiJohnandjohn 75c71390d5 [Bugfix] Escape control characters in xgrammar choice grammar (#48115)
Signed-off-by: john <6712432+CaiJohn@users.noreply.github.com>
Co-authored-by: john <6712432+CaiJohn@users.noreply.github.com>
2026-09-17 15:57:47 +03:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +00:00
yzong-rh e6960af33b [Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200) 2026-09-15 09:48:49 -04:00
cd10ed6f9f [feature] [xgrammar] support patternProperties/propertyNames/unevaluatedProperties kw for object types (#42904)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 12:58:52 +08:00
Julien Denize 983b7e28c9 [Bugfix] Avoid MistralCommonBackend for HF tokenizers (#54192)
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
2026-09-10 17:57:44 +00:00
c55e15a44e [Structured Output] Keep invalid structured-output requests from stopping the engine (#51450)
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com>
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
2026-09-09 15:32:01 +00:00
Flora FengandWentao Ye 29f46ac475 [Perf] Eliminate full-history reasoning scans for structured outputs (#55223)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
2026-09-08 14:07:16 -07:00
Chauncey c6e19b3be2 [Bugfix][Structured Output] Avoid spurious FSM errors after speculative reasoning end (#53046)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
2026-08-21 10:01:06 +08:00
Thanh Phan d56bbf3995 [Bugfix] Support MistralCommonBackend tokenizers in structured output (#52720)
Signed-off-by: Thanh Phan <thanhphantuan1110@gmail.com>
2026-08-20 19:08:50 +01:00
Yufeng He 340b7e4909 fix: reject string schemas that mix pattern/format with length bounds (#49996)
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
2026-08-19 17:29:32 +08:00
Flora Feng 12f64b39d2 [Bugfix][Structured Output] Stop XGrammar token batches at termination (#52805)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-08-18 21:21:43 +00:00
5ae2d38821 [Perf][Structured Output] Skip unused request-local reasoners (#52573)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-08-17 16:30:41 -07:00
Jeffrey WangandClaude Opus 5 41f12a0daf [Bugfix] Raise VLLMValidationError from structured output validators (#52394)
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 01:04:51 -07:00
Junhao ShenandClaude Opus 4.8 38f097fab8 [Bugfix] Reject NUL byte in structured_outputs.regex (#51796)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-14 00:08:36 +00:00
Flora Feng 05f0a80016 [Bugfix][Structured Output] Mask request stop tokens in xgrammar until grammar terminates (#49227)
Signed-off-by: sfeng33 <4florafeng@gmail.com>
2026-08-10 14:29:20 -04:00
Chang GuoandClaude Opus 4.8 b91a40e729 [Bugfix] Restore structured output logger initialization (#49626)
Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 11:26:03 -07:00
yue.yu 0416dab275 [Bugfix][Structured Output][Spec Decode] Advance grammar across reasoning boundary (#44993)
Signed-off-by: Allen.Yu <yuyue0225sc@163.com>
2026-07-23 09:14:15 -07:00
zhrrrandNick Hill 12213c6795 [Bugfix] handle grammar compilation failures to avoid engine crash (#47312)
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
2026-07-23 18:13:49 +08:00
Nick Hill af91f4b3e4 [Cleanup] Remove unused StructuredOutputRequest.status field (#49235)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-07-21 00:06:16 +00:00
Russell BryantandClaude 96d2ceda4b [Security] Replace diskcache to eliminate pickle deserialization (#44549)
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-14 20:29:24 -07:00
Juan Pérez de Algaba c9a788eedc fix(security): guard lm-format-enforcer regex compile with timeout (#47595)
Signed-off-by: jperezde <jperezde@redhat.com>
2026-07-14 07:18:11 +00:00
Harry Mellor dd94484577 Bump Transformers version to 5.10.4 (#41359)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-07-07 05:13:28 -07:00
yue.yuandBenjamin Chislett e7c9df9449 [Bugfix][Structured Output][Spec Decode] Constrain bitmask and trim grammar advance at the reasoning boundary (#44297)
Signed-off-by: Allen.Yu <yuyue0225sc@163.com>
Signed-off-by: yue.yu <yuyue0225sc@163.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>
2026-07-04 09:08:45 +00:00
Nick Hill 7df3d7dada [Core] Ensure memory is pinned prior to async h2d copy (#45424)
Signed-off-by: Nick Hill <nickhill123@gmail.com>
2026-06-20 20:02:24 -07:00
Juan Pérez de AlgabaandHarry Mellor 2b3006076c [Security] Add timeout guard for regex compilation in structured outp… (#45118)
Signed-off-by: jperezde <jperezde@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
2026-06-13 09:52:56 +00:00
eb28452b10 [Model] Add DiffusionGemma Support (#45163)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Martin Kukla <martin.kukla@cantab.net>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Dipika Sikka <dsikka@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com>
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com>
2026-06-11 22:17:35 -07:00
rishitdholakia13andCursor ded871201a [Bug][Structured Outputs] Fix bug that leads to unconstrained generations with structural tags (#42452)
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 07:08:58 -07:00
Bugen Zhaoandwang.yuqi a07642667d [Bugfix] Pass reasoning parser kwargs to structured output (#41199)
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
2026-04-30 23:38:02 -07:00
Julien Denize fef56c1855 [Mistral Grammar] Support Grammar Factory (#38150)
Signed-off-by: juliendenize <julien.denize@mistral.ai>
2026-04-06 10:28:51 -04:00
Jeffrey Wang 2d725b89c5 [Bugfix] Lazy import diskcache to avoid sqlite3/libstdc++ ImportError at startup (#38649)
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
2026-04-01 05:31:20 +00:00