MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
4ac1f8d2ac
[Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends ( #61574 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 12:53:41 +08:00
818f70c295
[Bugfix] Mask special added tokens in xgrammar JSON, regex and EBNF grammars ( #60749 )
...
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-13 14:38:23 +03:00
samayp01 and Artem Perevedentsev
54db759eea
[Bugfix][Structured Output] Reject structural_tag and unhandled specs in backend validation ( #58862 )
...
Signed-off-by: Samay <samay.piduru@gmail.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-13 13:27:34 +03:00
DimensionSTP and Artem Perevedentsev
314ab8cffd
[Bugfix][Structured Output] Drain compilation tasks before releasing the backend ( #61047 )
...
Signed-off-by: DimensionSTP <65501090+DimensionSTP@users.noreply.github.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-12 16:25:36 +03:00
Kai Hoang and Flora Feng
e0cb3efce0
[Bugfix][Structured Output] Decode choice specs as JSON in outlines and LMFE ( #60748 )
...
Signed-off-by: hpdkhoa <hpdkhoa2311@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-09 20:52:08 +00:00
Willow Lopez and Artem Perevedentsev
ad8a6065f4
[Bugfix] Validate guidance tokenizer compatibility before EngineCore ( #52298 )
...
Signed-off-by: Oxygen56 <jiangth99@163.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 21:59:02 +03:00
Shreyansh Jain and Artem Perevedentsev
28d76f3bad
[Bugfix][Structured Output] Reject unsupported regex inside groups ( #60359 )
...
Signed-off-by: Shreyansh Jain <shreyanshjain646@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 14:16:13 +03:00
f43e4c39e3
[Structured Output] Allow propertyNames with additionalProperties on xgrammar ( #57594 )
...
Signed-off-by: errmakov <ide404@gmail.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-09 10:49:47 +03:00
c23ca06b6d
[Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar ( #59627 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com >
2026-10-08 19:38:33 +00:00
73c742b09c
[Bugfix] Make disable_any_whitespace actually disable whitespace on xgrammar ( #58067 )
...
Signed-off-by: Stu Cao <stu@primeintellect.ai >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-08 14:41:14 +03:00
Harry Mellor and Claude Opus 5.5
21d9758459
Remove Python 3.10 support now that it is EOL ( #60402 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 02:30:52 -07:00
Artem Perevedentsev and Shashank Varma
049507aa76
[Bugfix][Structured Output] Flag multi-branch allOf as unsupported for xgrammar ( #59061 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Shashank Varma <324153016+shashankvarma499@users.noreply.github.com >
2026-10-06 17:01:04 +04:00
Mai Hoàng Đức and Flora Feng
5cd8a2626b
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls ( #60036 )
...
Signed-off-by: Mazukiri <maihoangduc04@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 00:06:59 +00:00
60932a6401
[Bugfix][Structured Output] Preserve literal values in Guidance disable_additional_properties ( #58709 )
...
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com >
Co-authored-by: Antigravity <noreply@google.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-05 18:12:40 +03:00
Shinichi Hemmi
17cc3cd80e
[Model] Engine based plamo3 parser ( #56575 )
...
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp >
2026-09-29 05:17:42 +00:00
1f0bc49ee6
[Structured Outputs] Parse Lark grammars natively in the xgrammar backend ( #58321 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-25 04:35:10 +00:00
SIDDARTHA REDDY
a811738a60
[Bugfix] Accept EOS after grammar finish in outlines backend; reject json_object at validation ( #57743 )
...
Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com >
2026-09-24 17:17:10 -07:00
Nick Hill
36f94d5fe8
[Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts ( #58612 )
2026-09-24 21:55:56 +00:00
9815732c16
[BugFix][Core] Make the structured-output grammar poll non-blocking ( #55931 )
...
Signed-off-by: ubwzwd <ubwzwd@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-09-22 14:39:47 +03:00
3c84ad13ed
[Bugfix] Fix xgrammar feature gate bypass when JSON Schema type is a list ( #48416 )
...
Signed-off-by: john <6712432+CaiJohn@users.noreply.github.com >
Co-authored-by: john <6712432+CaiJohn@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
2026-09-21 09:59:20 +03:00
CaiJohn and john
75c71390d5
[Bugfix] Escape control characters in xgrammar choice grammar ( #48115 )
...
Signed-off-by: john <6712432+CaiJohn@users.noreply.github.com >
Co-authored-by: john <6712432+CaiJohn@users.noreply.github.com >
2026-09-17 15:57:47 +03:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
yzong-rh
e6960af33b
[Refactor] StructuredOutputManager x Speculative Decoding Refactor ( #48200 )
2026-09-15 09:48:49 -04:00
cd10ed6f9f
[feature] [xgrammar] support patternProperties/propertyNames/unevaluatedProperties kw for object types ( #42904 )
...
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-09-15 12:58:52 +08:00
Julien Denize
983b7e28c9
[Bugfix] Avoid MistralCommonBackend for HF tokenizers ( #54192 )
...
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com >
2026-09-10 17:57:44 +00:00
c55e15a44e
[Structured Output] Keep invalid structured-output requests from stopping the engine ( #51450 )
...
Co-authored-by: Lucas Bourtoule <35483370+dhalf@users.noreply.github.com >
Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com >
2026-09-09 15:32:01 +00:00
Flora Feng and Wentao Ye
29f46ac475
[Perf] Eliminate full-history reasoning scans for structured outputs ( #55223 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
2026-09-08 14:07:16 -07:00
Chauncey
c6e19b3be2
[Bugfix][Structured Output] Avoid spurious FSM errors after speculative reasoning end ( #53046 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-08-21 10:01:06 +08:00
Thanh Phan
d56bbf3995
[Bugfix] Support MistralCommonBackend tokenizers in structured output ( #52720 )
...
Signed-off-by: Thanh Phan <thanhphantuan1110@gmail.com >
2026-08-20 19:08:50 +01:00
Yufeng He
340b7e4909
fix: reject string schemas that mix pattern/format with length bounds ( #49996 )
...
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
2026-08-19 17:29:32 +08:00
Flora Feng
12f64b39d2
[Bugfix][Structured Output] Stop XGrammar token batches at termination ( #52805 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-08-18 21:21:43 +00:00
5ae2d38821
[Perf][Structured Output] Skip unused request-local reasoners ( #52573 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-08-17 16:30:41 -07:00
Jeffrey Wang and Claude Opus 5
41f12a0daf
[Bugfix] Raise VLLMValidationError from structured output validators ( #52394 )
...
Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-16 01:04:51 -07:00
Junhao Shen and Claude Opus 4.8
38f097fab8
[Bugfix] Reject NUL byte in structured_outputs.regex ( #51796 )
...
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-08-14 00:08:36 +00:00
Flora Feng
05f0a80016
[Bugfix][Structured Output] Mask request stop tokens in xgrammar until grammar terminates ( #49227 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-08-10 14:29:20 -04:00
Chang Guo and Claude Opus 4.8
b91a40e729
[Bugfix] Restore structured output logger initialization ( #49626 )
...
Signed-off-by: Change72 <changg@nvidia.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-23 11:26:03 -07:00
yue.yu
0416dab275
[Bugfix][Structured Output][Spec Decode] Advance grammar across reasoning boundary ( #44993 )
...
Signed-off-by: Allen.Yu <yuyue0225sc@163.com >
2026-07-23 09:14:15 -07:00
zhrrr and Nick Hill
12213c6795
[Bugfix] handle grammar compilation failures to avoid engine crash ( #47312 )
...
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com >
Signed-off-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
2026-07-23 18:13:49 +08:00
Nick Hill
af91f4b3e4
[Cleanup] Remove unused StructuredOutputRequest.status field ( #49235 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-07-21 00:06:16 +00:00
Russell Bryant and Claude
96d2ceda4b
[Security] Replace diskcache to eliminate pickle deserialization ( #44549 )
...
Signed-off-by: Russell Bryant <rbryant@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-07-14 20:29:24 -07:00
Juan Pérez de Algaba
c9a788eedc
fix(security): guard lm-format-enforcer regex compile with timeout ( #47595 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
2026-07-14 07:18:11 +00:00
Harry Mellor
dd94484577
Bump Transformers version to 5.10.4 ( #41359 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-07 05:13:28 -07:00
yue.yu and Benjamin Chislett
e7c9df9449
[Bugfix][Structured Output][Spec Decode] Constrain bitmask and trim grammar advance at the reasoning boundary ( #44297 )
...
Signed-off-by: Allen.Yu <yuyue0225sc@163.com >
Signed-off-by: yue.yu <yuyue0225sc@163.com >
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com >
2026-07-04 09:08:45 +00:00
Nick Hill
7df3d7dada
[Core] Ensure memory is pinned prior to async h2d copy ( #45424 )
...
Signed-off-by: Nick Hill <nickhill123@gmail.com >
2026-06-20 20:02:24 -07:00
Juan Pérez de Algaba and Harry Mellor
2b3006076c
[Security] Add timeout guard for regex compilation in structured outp… ( #45118 )
...
Signed-off-by: jperezde <jperezde@redhat.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-06-13 09:52:56 +00:00
eb28452b10
[Model] Add DiffusionGemma Support ( #45163 )
...
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com >
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Martin Kukla <martin.kukla@cantab.net >
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com >
Co-authored-by: Dipika Sikka <dsikka@redhat.com >
Co-authored-by: NickLucche <nlucches@redhat.com >
Co-authored-by: jiahanc <173873397+jiahanc@users.noreply.github.com >
Co-authored-by: Alec Kohlhoff <134344302+aleckohlhoff@users.noreply.github.com >
Co-authored-by: Porras Huang <20535584+porrashuang@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: scoootscooob <167050519+scoootscooob@users.noreply.github.com >
2026-06-11 22:17:35 -07:00
rishitdholakia13 and Cursor
ded871201a
[Bug][Structured Outputs] Fix bug that leads to unconstrained generations with structural tags ( #42452 )
...
Signed-off-by: rishitdholakia13 <rishit+github@cohere.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-05-20 07:08:58 -07:00
Bugen Zhao and wang.yuqi
a07642667d
[Bugfix] Pass reasoning parser kwargs to structured output ( #41199 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-04-30 23:38:02 -07:00
Julien Denize
fef56c1855
[Mistral Grammar] Support Grammar Factory ( #38150 )
...
Signed-off-by: juliendenize <julien.denize@mistral.ai >
2026-04-06 10:28:51 -04:00