iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
22,313 Commits 3 Branches 0 Tags
c4fc3c7d322bbcd1b25307ccbd9770c8eebc5c09
Commit Graph
83 Commits
Author SHA1 Message Date
Yoni Gozlan 2eaa3bc5ac [Frontend] Enrich chat_parsing streaming events (#58603) 
Give serving adapters the delimiter spans and opener captures they need,
and keep streaming after a region that fails to parse.

Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Fix mypy index types in the event-stream test.
 2026-10-01 11:13:13 +08:00
Yoni GozlanYoni GozlanCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		73a5831127 [Frontend] Port chat_parsing core from Transformers (#58602) 
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 00:21:02 +08:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
yzong-rh 1d331f8341 [Bugfix] Suppress HarmonyError Unexpected token while expecting start token 200006 (#59254) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-09-29 16:40:15 -04:00
Yixin DongCodexyuchuanBugen ZhaoClaude Opus 5.5
 		dfc8e0f3e2 [Frontend] Support strict MiMo-V2.6 tool calling (#58019) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 12:47:47 +00:00
Václav ČadekandCursor Agent 6cbbea8721 [Bugfix] Hoist $defs/definitions in Cohere parser tool schema composition (#49602) 
Signed-off-by: Václav Čadek <vaclavcadek@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-09-29 07:11:56 +00:00
Shinichi Hemmi 17cc3cd80e [Model] Engine based plamo3 parser (#56575) 
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp>
 2026-09-29 05:17:42 +00:00
Artem PerevedentsevYufeng HepablopupohubuntClaude Sonnet 5
 		2b9b55c7f1 [Bugfix] Support repsonse_format + tool_choice=auto (#56086) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com>
Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-27 21:17:34 +04:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 09:16:53 -04:00
Baljinder HothiandFlora Feng fdcce47e9b [Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792) 
Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-26 00:38:52 +00:00
Yousafandsfeng33 83db8c8390 [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) 
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-09-25 20:43:08 +00:00
Samyabrata MajiandFlora Feng 064bc747ab [Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626) 
Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 00:12:29 +00:00
Nikhil KulkarniClaude Opus 4.8Chauncey
 		15ed1262e7 [Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine (#49648) 
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 10:37:40 +00:00
lazymioClaude Opus 5Bugen Zhao
 		39e33db7f3 [Frontend] Fix the parsing of missing string= in DeepSeek V4 (#56271) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
 2026-09-18 05:11:24 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
dev 8be5205abb [Parser] Fix: correct parser frontend handling of reasoning end and boundary tokens (#56635) 
Signed-off-by: DEV TYAGI <devtyagi3909@gmail.com>
 2026-09-16 01:17:22 -07:00
yzong-rh e6960af33b [Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200) 2026-09-15 09:48:49 -04:00
Flora Feng 24bfbd5d4d [Refactor] Derive is_reasoning_end from the engine grammar (#56200) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-15 10:07:58 +00:00
Walter Beller-MoralesandFlora Feng 756794a9a7 [Frontend] create unified Cohere parser (#56392) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-12 04:04:46 -04:00
Roger WangKimi Codezjy0516Codex
 		9b959b8657 [Model] DeepSeek-V4.1-Flash Model Definitions (#56228) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-10 20:16:26 +08:00
zexplorerhjCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fe26c705f7 [Model] Add Bailing V3 VL support (#55921) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-10 02:46:56 -07:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 01:15:06 -07:00
Flora Feng 9e25706560 [Bugfix] Tolerate misspelled DSML tool_calls wrapper (#56141) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-10 15:04:29 +08:00
Mikhail PodvitskiiandFlora Feng 65f3fca568 [Bugfix][Parser] Seed-OSS turn-boundary tokens + boundary-fallback tests (#54264) 
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-09 23:29:30 +00:00
Flora Feng d98c8c030b [Bugfix] Parse DSML tool calls when the model omits the tool_calls wrapper (#55954) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-08 17:38:04 -07:00
Flora FengandWentao Ye 29f46ac475 [Perf] Eliminate full-history reasoning scans for structured outputs (#55223) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-08 14:07:16 -07:00
Flora Feng 300f688321 [Bugfix] Implicitly close DeepSeek DSML parameters (#54838) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-02 14:13:36 +08:00
Suvrakamal DasClaudeBen Browning
 		e7cf4730d6 [Bugfix] Drop incomplete tool-call markup in non-streaming to match streaming (#47562) 
Signed-off-by: Suvrakamal Das <jyb2084.usa@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-01 18:11:36 +00:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-31 10:13:50 +00:00
Mikhail Podvitskii 3a9bfc209f [Bugfix][Parser] Scope reasoning-end detection to the current turn via turn-boundary tokens (#54089) 
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
 2026-08-27 16:57:32 -04:00
yzong-rhandBen Browning 51d1b45182 [Bugfix][GPT-OSS] Fix strict tool-call grammar to accept Harmony renders (#52222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-08-26 18:05:09 -07:00
Flora Feng 46638857fd [Bugfix][Structured Output] Preserve reasoning adapters for shared parser engines (#52830) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-08-26 00:37:36 -04:00
Alexander LeeChaunceymergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		d29dc3ab87 [Bugfix][Gemma4] Align parser enable_thinking default with template (#52430) 
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-18 12:41:26 +00:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ac2ae8798c [Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-15 06:51:14 -07:00
yzong-rh e6b2a8ad5e [Bugfix][Structured Output] Mask request stop tokens in xgrammar until grammar terminates (#50595) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-13 14:07:59 -04:00
Flora Feng 3f142bd85e [Bugfix] Align deepseek v4 parser thinking default with tokenizer (#51296) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-08-10 19:30:30 +00:00
Jason 0820125ae9 [Bugfix][Parser] Emit REASONING_END for Inkling tool calls that follow no thinking block (#50528) 
Signed-off-by: Jason Yao <wsyjh8@gmail.com>
 2026-08-09 21:04:53 -04:00
Vegetog 1c1077c6cc [Bugfix][Parser] Confirm reasoning end when an Inkling content block opens (#49876) 
Signed-off-by: Vegetog <110553275+Vegetog@users.noreply.github.com>
 2026-08-08 18:07:39 -04:00
cherry77-cloudBen BrowningClaude Opus 4.8
 		75231eff2f [Bugfix][Parser] Prevent Inkling block-end leakage with tools (#51391) 
Signed-off-by: cherry77-cloud <1615405@qq.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-08 12:34:32 -04:00
zexplorerhj d4da0c55af [Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support (#51045) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
 2026-08-06 00:38:28 +08:00
yzong-rh dc818c198d [GPT-OSS] Strict tool call and constrained decoding for Harmony (#45560) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-01 12:10:26 -04:00
Bugen Zhao 629a938a92 [Frontend] Preserve bare Inkling text in Python and Rust parsers (#50403) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-30 12:25:10 -07:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
yzong-rh 3f47a8384d [Bugfix] Fix VLLM_ENFORCE_STRICT_TOOL_CALLING mutation in tests (#49846) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-07-27 16:12:14 +00:00
Ben Browningandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 3de4b2bf3c [Bugfix][Parser] Fix special tokens (EOS/BOS) leaking into reasoning content (#48748) 
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-22 16:10:55 -04:00
mosya415mosya415Ben Browning
 		11d291511a [Bugfix][Tool Parser] Preserve whitespace in parameter values (MiniMax M2, Qwen3, MiniCPM5 XML) (#48846) 
Signed-off-by: mosya415 <263250241+mosya415@users.noreply.github.com>
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: mosya415 <263250241+mosya415@users.noreply.github.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-17 16:45:41 +00:00
Woosuk KwonBugen ZhaoGiancarlo DelfinIsotr0pyIsotr0pyJee Jee LiRoger WangYifan QiaoClaude Fable 5OpenAI Codex
 		6570c9800c [Model] Add Inkling model support [1/N] (#48799) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-15 23:40:07 -07:00
adhi29andBen Browning af453e5647 [Bugfix] Gemma4 parser: classify channel-less output consistently in streaming and non-streaming (#48262) 
Signed-off-by: Adhithya Balakrishnan <adhithya.b2004@gmail.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-14 09:30:16 -04:00
akii96andBen Browning 550218b136 [Bugfix][Frontend] Flush engine reasoning parser at engine-reasoning → tool streaming boundary (#47606) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-13 14:06:10 -04:00