MOVZX and Hinata
2b544b7ba0
[Mini-VLLM] Cut dead subsystems across serving, connectors, parsers
...
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 02:19:03 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
153a17166e
[Frontend] Add the Beam reasoning and tool parsers ( #62052 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com >
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 14:08:29 -07:00
alberto
ae8f41238c
[Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts ( #61857 )
...
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com >
2026-10-16 12:54:42 -04:00
tc-mb and Cyrus Leung
c85616efc6
[Model] Support MiniCPM-V 4.7 ( #58674 )
...
Signed-off-by: tc-mb <tianchi_cai@icloud.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-10-11 10:21:10 +00:00
Misha Goin and Claude
1ab18599ca
[Agents] Add pre-commit check for agent files and skills ( #59657 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-02 09:38:22 -07:00
Harry Mellor and Claude Opus 5.5
5d20979fc9
[Misc] Remove code paths for Transformers < 5.16.1 ( #59762 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 11:17:24 +00:00
9bd993564f
[Frontend] Parse tool calls and reasoning from checkpoint response templates ( #58604 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-10-02 02:33:00 +00:00
Flora Feng
a5105dba05
[Frontend] Port Step-3.5 parsers to the streaming parser engine ( #59321 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-10-01 22:18:37 +00:00
dfc8e0f3e2
[Frontend] Support strict MiMo-V2.6 tool calling ( #58019 )
...
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Signed-off-by: Ubospica <ubospica@gmail.com >
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-09-29 12:47:47 +00:00
Shinichi Hemmi
17cc3cd80e
[Model] Engine based plamo3 parser ( #56575 )
...
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp >
2026-09-29 05:17:42 +00:00
Elvir Crnčević and Flora Feng
9bf44c4013
[Bugfix][Reasoning] Count Kimi K3 reasoning tokens ( #58372 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 23:29:56 +00:00
Yousaf and sfeng33
83db8c8390
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 ( #55957 )
...
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-09-25 20:43:08 +00:00
Samyabrata Maji and Flora Feng
064bc747ab
[Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers ( #58626 )
...
Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 00:12:29 +00:00
Wentao Ye and Flora Feng
f3aa88d230
[Kimi K3 Bug] Fix kimi k3 reasoning parser ( #57098 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-17 10:05:52 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Flora Feng
3bb7826214
[Agent] Add agent instructions for parser directories ( #56883 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-09-16 06:55:26 -04:00
Walter Beller-Morales and Flora Feng
756794a9a7
[Frontend] create unified Cohere parser ( #56392 )
...
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-12 04:04:46 -04:00
Bugen Zhao and Isotr0py
b47b01cf33
[Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends ( #56208 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
2026-09-10 01:15:06 -07:00
yzong-rh
26fec6d183
[Bugfix] Fix Step-3.5 reasoning parser for structured outputs ( #53174 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-09-09 17:10:23 -04:00
JasonCohere and Cursor
bb363db9a5
feat: Add support for reasoning_token_count to reasoning parser ( #54982 )
...
Signed-off-by: Jason Ozuzu <jasonozuzu@cohere.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-03 05:02:49 -07:00
Bowen Tan
1f76efaa21
[Model] Add K2-Horizon model support ( #55063 )
2026-09-03 01:17:08 -07:00
Yusuke Oda and Chauncey
798b557e06
[Frontend] Add site-packages support for reasoning/tool parser plugins ( #45241 )
...
Signed-off-by: odashi <yusuke.oda@predicate.jp >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-02 07:41:13 +00:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
b2f685834a
[Hy4] support Hy4-preview model ( #54160 )
...
Signed-off-by: chengvjiang <chengvjiang@tencent.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: chengvjiang <chengvjiang@tencent.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: russellfeng <russellfeng@tencent.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-08-29 12:07:28 +08:00
andrewbcohere
f94dcde5c5
Fix Cohere ChatV2 citation and tool handling issues ( #52175 )
...
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com >
2026-08-21 15:23:17 -07:00
6adad08767
Add Muse Glimmer model support ( #51655 )
...
Signed-off-by: Tiezhen Wang <tiezhen@inferact.ai >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Signed-off-by: zixi-qi <zixi@inferact.ai >
Co-authored-by: Beto de Paola <betodepaola@meta.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: zixi-qi <zixi@inferact.ai >
2026-08-14 12:28:56 +08:00
JasonCohere
5fee0a872d
chore: Upstream Cohere parser fixes + tests ( #51998 )
...
Signed-off-by: Jason Ozuzu <jasonozuzu@cohere.com >
2026-08-13 05:05:36 -07:00
zexplorerhj
d4da0c55af
[Model][Frontend] Add Ling 3.0 Flash BF16, MTP, and parser support ( #51045 )
...
Signed-off-by: zexplorerhj <zhjoneson@163.com >
2026-08-06 00:38:28 +08:00
Bowen Wang and Claude Opus 5
adbf08d977
[Bugfix][Reasoning] kimi_k3: O(delta) reasoning-end check on the decode path ( #50886 )
...
Signed-off-by: abmfy <abmfy@icloud.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-04 10:37:03 +08:00
yzong-rh
dc818c198d
[GPT-OSS] Strict tool call and constrained decoding for Harmony ( #45560 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-08-01 12:10:26 -04:00
andrewbcohere and Cyrus Leung
9c110fa522
[Frontend] Cohere chat v2 api support ( #47189 )
...
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-31 22:53:54 -07:00
Julien Denize
1a20d23dab
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls ( #48947 )
2026-07-30 15:38:00 +02:00
Bugen Zhao
f5a7cce9b6
[Model] Add Kimi K3 support: Python frontend [2/2] ( #50093 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-29 01:06:01 -07:00
Wentao Ye
b5bcb3ce88
[Refactor] Remove dead code in multiple files ( #49745 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
2026-07-27 15:58:26 -04:00
6570c9800c
[Model] Add Inkling model support [1/N] ( #48799 )
...
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: Isotr0py <mozf@inferact.ai >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-07-15 23:40:07 -07:00
Harry Mellor
dd94484577
Bump Transformers version to 5.10.4 ( #41359 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-07-07 05:13:28 -07:00
Ben Browning
fb5291b35b
[Frontend] [Parser] Port DeepSeek V4 to streaming parser engine framework ( #45877 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
2026-07-03 20:55:23 -04:00
stevenkuang
cc56379e28
[Model] Support Hy3 token suffix and JSON Schema array types ( #47192 )
...
Signed-off-by: stevenkuang-tencent <stevenkuang@tencent.com >
2026-07-01 10:16:07 +00:00
hcl and Flora Feng
9294dd27eb
fix(reasoning): guard rfind in ernie45 streaming </response> branch ( #46255 )
...
Signed-off-by: Chenglun Hu <chenglunhu@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-07-01 01:01:14 +00:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2bc20e8aba
[Frontend] Add Streaming Parser Engine and new Kimi k2.5/k2.6/k2.7 Parser ( #46610 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-30 07:53:17 +00:00
d706dec904
fix: Correct reasoning-end detection for prompt history ( #44551 )
...
Signed-off-by: jwzheng96 <jianweizheng@pku.edu.cn >
Signed-off-by: JianweiZheng <32029023+jwzheng96@users.noreply.github.com >
Signed-off-by: Jason Ozuzu <jasonozuzu@cohere.com >
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: JianweiZheng <32029023+jwzheng96@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: Walter Beller-Morales <walterbm@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-06-26 22:15:06 -07:00
Maxwill Lin
cd347298e8
[Frontend] Port seed_oss to the streaming parser engine as a Qwen3 subclass ( #46314 )
...
Signed-off-by: EazyReal <8047065+EazyReal@users.noreply.github.com >
2026-06-24 20:08:42 -04:00
Rui Yin
d8e422ccda
[Bugfix] Parse MiniMax M3 streaming reasoning by text markers ( #45718 )
...
Signed-off-by: test test <2260891073@qq.com >
2026-06-23 14:43:58 -04:00
Flora Feng
560fb8b867
[Cohere] Remove dead prepare_structured_tag override in Cohere parser ( #46099 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-06-19 01:02:11 +00:00
Ivy Xu
c3c6d723fd
[Perf] Remove unused loggers in reasoning/ ( #45988 )
...
Signed-off-by: Ivy <fakeshadow1337@gmail.com >
2026-06-18 22:24:29 +00:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
6c379b9e54
[Frontend] Add Streaming Parser Engine and new GLM4.7/GLM5.1/GLM5.2 Parser ( #45915 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-19 00:42:10 +08:00
Nikhilesh Chhetri
3c6084bb0d
[Bugfix][Gemma4] Pre-initialise streaming reasoning state when prompt ends inside an open <|channel> ( fixes #45834 ) ( #45852 )
...
Signed-off-by: nikhilesh-csa <nchhetri@csa1.com >
2026-06-17 06:16:02 -04:00
Chauncey
f00e163f35
[Frontend] Add Streaming Parser Engine and new MinimaxM2 Parser ( #45701 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-06-16 13:38:17 -04:00
Ben Browning
f3858d5422
[Frontend] [Parser] Migrate Nemotron V3 to streaming parser engine ( #45755 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
2026-06-16 05:31:21 +00:00