Afonso Ribeiro and Flora Feng
5c1c903501
[Perf][Parser] Stream Qwen3 tool argument values incrementally (12 KB call: 2.1 s to 11 ms) ( #62027 )
...
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-17 18:49:06 +00:00
Afonso Ribeiro and Flora Feng
b088eba944
[Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) ( #62035 )
...
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-16 22:46:02 +00:00
153a17166e
[Frontend] Add the Beam reasoning and tool parsers ( #62052 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 14:08:29 -07:00
alberto
ae8f41238c
[Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts ( #61857 )
...
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com >
2026-10-16 12:54:42 -04:00
4ac1f8d2ac
[Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends ( #61574 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com >
2026-10-16 12:53:41 +08:00
Bugen Zhao and Claude Opus 5.5
8fa5ae844a
[Frontend] Let an unset --tool-strict-level fall back to the tool parser's default ( #61328 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:48:38 +08:00
Ben Browning and Flora Feng
4c5ea86f53
[Bugfix][Parser] Stream Gemma4 string tool arguments incrementally ( #61494 )
...
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-14 21:25:41 +00:00
3f466c852c
[Bugfix][Parser] Recover genuine tool calls after unclosed Qwen3 <function= opener in prose ( #59928 )
...
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com >
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Gemini <noreply@google.com >
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com >
2026-10-14 13:01:25 +03:00
Yoni Gozlan and Bugen Zhao
69d00bc264
[Bugfix] Accept whole-number floats for integer tool arguments in the response parser ( #61297 )
...
Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
2026-10-13 12:55:20 +08:00
Maroon Ayoub and Flora Feng
44f0ddfd7b
[Bugfix][Parser] Stop re-parsing the whole GLM tool argument per chunk ( #61253 )
...
Signed-off-by: Maroon Ayoub <mayoub@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-12 21:28:35 +00:00
Shijin Zhang and Chauncey
83773ecdf3
[Frontend] Support CustomToolCall in the non-Harmony Responses API ( #57322 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-10-12 17:22:55 +08:00
tc-mb and Cyrus Leung
c85616efc6
[Model] Support MiniCPM-V 4.7 ( #58674 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 10:21:10 +00:00
wang.yuqi
c1692c85f1
[Misc] mv chat_utils.py to renderers. ( #60522 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
2026-10-10 06:50:34 +00:00
c23ca06b6d
[Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar ( #59627 )
...
Signed-off-by: Simon Veitner <sveitner@redhat.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com >
2026-10-08 19:38:33 +00:00
Junyi Yao and sfeng33
fdfc171a74
[Bugfix][ToolParser] Stream every tool call when one delta completes several ( #60354 )
...
Signed-off-by: Junyi Yao <j.yao@wustl.edu >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-08 01:07:35 +00:00
www6v and sfeng33
38dc8ee500
[Bugfix] Validate tool-parser/tokenizer compatibility at startup ( #59749 )
...
Signed-off-by: www6v <www6v@126.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-07 23:46:41 +00:00
Dhairyashil R G and sfeng33
d6fe5dca68
[Bugfix][Parser] Emit buffered post-reasoning text when no tool parser is configured ( #58911 )
...
Signed-off-by: Dhairyashil R G <18738735+dhairyashilRG@users.noreply.github.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-10-07 23:21:51 +00:00
Bugen Zhao and Claude Opus 5.5
5145e35e50
[Frontend] Port the MiniMax M3 tool parser to the parser engine ( #59743 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 05:46:27 +00:00
lzhan011 and Flora Feng
a3e0243b1c
[Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON ( #54844 )
...
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 03:37:30 +00:00
Mai Hoàng Đức and Flora Feng
5cd8a2626b
[Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls ( #60036 )
...
Signed-off-by: Mazukiri <maihoangduc04@gmail.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-06 00:06:59 +00:00
RomeJay and Flora Feng
2e06395fd3
[Bugfix][Frontend] Build tool-call grammars from the tools the prompt renders ( #59879 )
...
Signed-off-by: Rongjie Jiang <j1870831863@outlook.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-05 22:01:03 +00:00
Shijin Zhang and Flora Feng
d1a974c140
[Frontend] Constrain non-strict GLM-4.7 tool calls with a shallow structural tag ( #56403 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-10-02 14:31:53 -04:00
Misha Goin and Claude
1ab18599ca
[Agents] Add pre-commit check for agent files and skills ( #59657 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-02 09:38:22 -07:00
9bd993564f
[Frontend] Parse tool calls and reasoning from checkpoint response templates ( #58604 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 02:33:00 +00:00
Flora Feng
a5105dba05
[Frontend] Port Step-3.5 parsers to the streaming parser engine ( #59321 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-10-01 22:18:37 +00:00
Yoni Gozlan
2eaa3bc5ac
[Frontend] Enrich chat_parsing streaming events ( #58603 )
...
Give serving adapters the delimiter spans and opener captures they need,
and keep streaming after a region that fails to parse.
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Fix mypy index types in the event-stream test.
2026-10-01 11:13:13 +08:00
73a5831127
[Frontend] Port chat_parsing core from Transformers ( #58602 )
...
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Co-authored-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 00:21:02 +08:00
Ostring24 and Flora Feng
4e7af3b0de
[Bugfix][Frontend] Enforce parallel_tool_calls=false in the required-tool grammar ( #50502 )
...
Signed-off-by: ostring <ostring@qq.com >
Signed-off-by: sfeng33 <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-29 21:33:28 +00:00
yzong-rh
1d331f8341
[Bugfix] Suppress HarmonyError Unexpected token while expecting start token 200006 ( #59254 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-09-29 16:40:15 -04:00
Shijin Zhang and Chauncey
d95d1dcfb9
[Bugfix][Frontend] Force reasoning mode for GLM-5.3 chat templates in the GLM MoE parser ( #56994 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-29 21:26:34 +08:00
dfc8e0f3e2
[Frontend] Support strict MiMo-V2.6 tool calling ( #58019 )
...
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Ubospica <ubospica@gmail.com >
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-29 12:47:47 +00:00
Václav Čadek and Cursor Agent
6cbbea8721
[Bugfix] Hoist $defs/definitions in Cohere parser tool schema composition ( #49602 )
...
Signed-off-by: Václav Čadek <vaclavcadek@gmail.com >
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-09-29 07:11:56 +00:00
Shinichi Hemmi
17cc3cd80e
[Model] Engine based plamo3 parser ( #56575 )
...
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp >
2026-09-29 05:17:42 +00:00
2b9b55c7f1
[Bugfix] Support repsonse_format + tool_choice=auto ( #56086 )
...
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com >
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com >
Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com >
Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com >
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-09-27 21:17:34 +04:00
Baljinder Hothi and Flora Feng
fdcce47e9b
[Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning ( #58792 )
...
Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-26 00:38:52 +00:00
Ricardo-M-L and Claude Opus 4.6
b040321b33
fix: perf: use startswith(x, i) instead of string slicing to avoid O(N^2) ( #52580 )
...
Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com >
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-09-25 23:22:13 +00:00
Yousaf and sfeng33
83db8c8390
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 ( #55957 )
...
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-09-25 20:43:08 +00:00
Samyabrata Maji and Flora Feng
064bc747ab
[Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers ( #58626 )
...
Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 00:12:29 +00:00
15ed1262e7
[Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine ( #49648 )
...
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com >
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-23 10:37:40 +00:00
39e33db7f3
[Frontend] Fix the parsing of missing string= in DeepSeek V4 ( #56271 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: lazymio <mio@lazym.io >
2026-09-18 05:11:24 +00:00
dee37d8911
[Frontend] Add --tool-strict-level for server-side control for structural tag activation ( #56268 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: lazymio <mio@lazym.io >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-18 04:32:39 +00:00
Flora Feng
e0705731be
[Refactor] Remove unused interface methods ( #57194 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-16 21:00:48 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Flora Feng
3bb7826214
[Agent] Add agent instructions for parser directories ( #56883 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-16 06:55:26 -04:00
dev
8be5205abb
[Parser] Fix: correct parser frontend handling of reasoning end and boundary tokens ( #56635 )
...
Signed-off-by: DEV TYAGI <devtyagi3909@gmail.com >
2026-09-16 01:17:22 -07:00
yzong-rh
e6960af33b
[Refactor] StructuredOutputManager x Speculative Decoding Refactor ( #48200 )
2026-09-15 09:48:49 -04:00
Flora Feng
24bfbd5d4d
[Refactor] Derive is_reasoning_end from the engine grammar ( #56200 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-15 10:07:58 +00:00
wuhangxian and Chauncey
a529c1a748
[Bugfix] Handle bare and malformed tool call openers in Gemma4 parser ( #53444 )
...
Signed-off-by: wuhangxian <1391938827@qq.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-15 02:52:43 +00:00
Walter Beller-Morales and Flora Feng
756794a9a7
[Frontend] create unified Cohere parser ( #56392 )
...
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-12 04:04:46 -04:00
