MOVZX and Hinata
2066d3143f
[Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack
...
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
cut deleted while keeping the model tree
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 17:10:56 +07:00
MOVZX and Hinata
2b544b7ba0
[Mini-VLLM] Cut dead subsystems across serving, connectors, parsers
...
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 02:19:03 +07:00
MOVZX and Hinata
54a396586e
[Mini-VLLM] Repair test suite after SM86 cut
...
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 01:06:30 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
Flora Feng
a5105dba05
[Frontend] Port Step-3.5 parsers to the streaming parser engine ( #59321 )
...
Signed-off-by: sfeng33 <4florafeng@gmail.com >
2026-10-01 22:18:37 +00:00
Harry Mellor and Claude Opus 5.5
fc2c801ad9
Disable mypy arg-type and assignment checks in tests ( #59428 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 15:18:50 +00:00
5225f1bacb
[MyPy][3/N] Fix MyPy errors in test groups (part 3) ( #55939 )
...
Signed-off-by: Taimys <ffy2281693949@gmail.com >
Signed-off-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: Luyi Xiao <81631911+luyixiao95@users.noreply.github.com >
Co-authored-by: Taimys <ffy2281693949@gmail.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 11:06:07 +00:00
Wentao Ye and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
834ad456d3
[Refactor] Remove dead tests code ( #58916 )
...
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-09-30 04:39:03 +00:00
Shijin Zhang and Chauncey
d95d1dcfb9
[Bugfix][Frontend] Force reasoning mode for GLM-5.3 chat templates in the GLM MoE parser ( #56994 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-09-29 21:26:34 +08:00
Elvir Crnčević and Flora Feng
9bf44c4013
[Bugfix][Reasoning] Count Kimi K3 reasoning tokens ( #58372 )
...
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com >
Signed-off-by: Flora Feng <4florafeng@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 23:29:56 +00:00
Yousaf and sfeng33
83db8c8390
[Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 ( #55957 )
...
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com >
Co-authored-by: sfeng33 <4florafeng@gmail.com >
2026-09-25 20:43:08 +00:00
Samyabrata Maji and Flora Feng
064bc747ab
[Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers ( #58626 )
...
Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-25 00:12:29 +00:00
Harry Mellor and Claude Opus 5
651a88c09a
[CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings ( #57212 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 10:57:24 -07:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Walter Beller-Morales and Flora Feng
756794a9a7
[Frontend] create unified Cohere parser ( #56392 )
...
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-09-12 04:04:46 -04:00
yzong-rh
26fec6d183
[Bugfix] Fix Step-3.5 reasoning parser for structured outputs ( #53174 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-09-09 17:10:23 -04:00
JasonCohere and Cursor
bb363db9a5
feat: Add support for reasoning_token_count to reasoning parser ( #54982 )
...
Signed-off-by: Jason Ozuzu <jasonozuzu@cohere.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-03 05:02:49 -07:00
Bowen Tan
1f76efaa21
[Model] Add K2-Horizon model support ( #55063 )
2026-09-03 01:17:08 -07:00
wang.yuqi and Cyrus Leung
eeb549a74d
[Frontend] Move engine/protocol.py out openai folder ( #54492 )
...
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io >
Signed-off-by: wang.yuqi <noooop@126.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
2026-08-31 10:13:50 +00:00
andrewbcohere
f94dcde5c5
Fix Cohere ChatV2 citation and tool handling issues ( #52175 )
...
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com >
2026-08-21 15:23:17 -07:00
d29dc3ab87
[Bugfix][Gemma4] Align parser enable_thinking default with template ( #52430 )
...
Signed-off-by: lxy-alexander <lxy_alexander@outlook.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-08-18 12:41:26 +00:00
JasonCohere
5fee0a872d
chore: Upstream Cohere parser fixes + tests ( #51998 )
...
Signed-off-by: Jason Ozuzu <jasonozuzu@cohere.com >
2026-08-13 05:05:36 -07:00
Bowen Wang and Claude Opus 5
adbf08d977
[Bugfix][Reasoning] kimi_k3: O(delta) reasoning-end check on the decode path ( #50886 )
...
Signed-off-by: abmfy <abmfy@icloud.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-08-04 10:37:03 +08:00
yzong-rh
dc818c198d
[GPT-OSS] Strict tool call and constrained decoding for Harmony ( #45560 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-08-01 12:10:26 -04:00
andrewbcohere and Cyrus Leung
9c110fa522
[Frontend] Cohere chat v2 api support ( #47189 )
...
Signed-off-by: Andrew Berneshawi <andrewberneshawi@cohere.com >
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk >
2026-07-31 22:53:54 -07:00
Julien Denize
1a20d23dab
[PARSER][Mistral] unified engine-based parser for reasoning and tool calls ( #48947 )
2026-07-30 15:38:00 +02:00
Bugen Zhao
f5a7cce9b6
[Model] Add Kimi K3 support: Python frontend [2/2] ( #50093 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
2026-07-29 01:06:01 -07:00
Ben Browning
fb5291b35b
[Frontend] [Parser] Port DeepSeek V4 to streaming parser engine framework ( #45877 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
2026-07-03 20:55:23 -04:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2bc20e8aba
[Frontend] Add Streaming Parser Engine and new Kimi k2.5/k2.6/k2.7 Parser ( #46610 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-30 07:53:17 +00:00
d706dec904
fix: Correct reasoning-end detection for prompt history ( #44551 )
...
Signed-off-by: jwzheng96 <jianweizheng@pku.edu.cn >
Signed-off-by: JianweiZheng <32029023+jwzheng96@users.noreply.github.com >
Signed-off-by: Jason Ozuzu <jasonozuzu@cohere.com >
Signed-off-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: JianweiZheng <32029023+jwzheng96@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: walterbm <walter.beller.morales@gmail.com >
Co-authored-by: Walter Beller-Morales <walterbm@users.noreply.github.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-06-26 22:15:06 -07:00
Maxwill Lin
cd347298e8
[Frontend] Port seed_oss to the streaming parser engine as a Qwen3 subclass ( #46314 )
...
Signed-off-by: EazyReal <8047065+EazyReal@users.noreply.github.com >
2026-06-24 20:08:42 -04:00
Rui Yin
d8e422ccda
[Bugfix] Parse MiniMax M3 streaming reasoning by text markers ( #45718 )
...
Signed-off-by: test test <2260891073@qq.com >
2026-06-23 14:43:58 -04:00
Chauncey and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
6c379b9e54
[Frontend] Add Streaming Parser Engine and new GLM4.7/GLM5.1/GLM5.2 Parser ( #45915 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-19 00:42:10 +08:00
Chauncey
f00e163f35
[Frontend] Add Streaming Parser Engine and new MinimaxM2 Parser ( #45701 )
...
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com >
2026-06-16 13:38:17 -04:00
Ben Browning
f3858d5422
[Frontend] [Parser] Migrate Nemotron V3 to streaming parser engine ( #45755 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
2026-06-16 05:31:21 +00:00
Luciano Martins and Luciano Martins
6607a80dab
[Bugfix][Gemma4] Fix offline parser truncation, adjust_request token leak, and chat template sync ( #45553 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
2026-06-16 04:31:53 +00:00
Ben Browning and Flora Feng
76a373eff4
[Frontend] Replace legacy Gemma4 parsers with engine-based implementation ( #45588 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-06-15 21:34:07 +00:00
0a1c5034f5
[Model] Add MiniMax M3 support ( #45381 )
...
Signed-off-by: youkaichao <youkaichao@gmail.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai >
Co-authored-by: OpenAI Codex <codex@openai.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai >
2026-06-16 01:01:25 +08:00
Andrii Skliar and Andrii Skliar
4882fd7632
[Bugfix][Reasoning] Nemotron V3: surface reasoning as content when thinking is unterminated ( #39091 )
...
Signed-off-by: Andrii Skliar <askliar@nvidia.com >
Co-authored-by: Andrii Skliar <askliar@nvidia.com >
2026-06-10 05:58:19 -07:00
712ad0286c
[Bugfix] KimiK2ReasoningParser: guard against buffered end-token in streaming ( #41068 )
...
Signed-off-by: Keyi Li <likey6688@gmail.com >
Co-authored-by: Keyi Li <likey6688@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-05-04 17:42:05 +00:00
yzong-rh and Flora Feng
0d4f714208
[Bugfix] Remove tokenizer encode/decode calls from Olmo3 reasoning parser ( #40855 )
...
Signed-off-by: Yifan <yzong@redhat.com >
Co-authored-by: Flora Feng <4florafeng@gmail.com >
2026-04-27 19:36:54 -07:00
+7
4d51588e23
[Feat] DeepSeek V4 Rebased ( #40860 )
...
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: qizixi <zixi@inferact.ai >
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com >
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com >
Co-authored-by: Yongye Zhu <yongye@inferact.ai >
Co-authored-by: Simon Mo <simon@inferact.ai >
Co-authored-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
Co-authored-by: Nick Hill <nickhill123@gmail.com >
Co-authored-by: Roger Wang <hey@rogerw.io >
Co-authored-by: Roy Wang <yasong.wang@inferact.ai >
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai >
Co-authored-by: youkaichao <youkaichao@gmail.com >
Co-authored-by: Zhewen Li <jerven.vllm@gmail.com >
Co-authored-by: Zijing Liu <liuzijing2014@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: qizixi <zixi@inferact.ai >
Co-authored-by: Zhewen Li <zhewenli@inferact.ai >
2026-04-26 18:31:08 -07:00
Bugen Zhao and gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
c662b4359e
[Bugfix] Avoid mutating chat_template_kwargs in HYV3ReasoningParser initialization ( #40713 )
...
Signed-off-by: Bugen Zhao <i@bugenzhao.com >
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-24 13:08:58 +08:00
Doug Campos
92762edc53
[Bugfix] Treat <tool_call> as implicit reasoning end in Qwen3 parser ( #35687 )
...
Signed-off-by: Doug Campos <qmx@qmx.me >
2026-04-24 09:10:04 +08:00
stevenkuang and Jee Jee Li
d0009ddb0b
[Model] Support Hy3 preview ( #40681 )
...
Signed-off-by: stevenkuang <stevenkuang@tencent.com >
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com >
2026-04-23 22:08:26 +08:00
Fergus and Chauncey
5544f8c18b
[Performance] Add is_reasoning_end_streaming() override to GptOssReasoningParser ( #35745 )
...
Signed-off-by: Fergus <fergus.barratt00@gmail.com >
Signed-off-by: fergus barratt <fergus.barratt00@gmail.com >
Co-authored-by: Chauncey <chaunceyjiang@gmail.com >
2026-04-21 18:31:27 +00:00
03f8d3a548
Update to transformers v5 ( #30566 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: khluu <khluu000@gmail.com >
Signed-off-by: Kevin H. Luu <khluu000@gmail.com >
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: khluu <khluu000@gmail.com >
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com >
Co-authored-by: jiang1.li <jiang1.li@intel.com >
2026-04-15 16:29:15 -07:00
8477fe427d
[Tool] adjust_request to reasoning parser, and Gemma4 fixes ( #39027 )
...
Signed-off-by: Ben Browning <bbrownin@redhat.com >
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-04-08 19:04:04 +00:00
08ed2b9688
feat(models): implement Google Gemma 4 architecture support (MoE, Multimodal, Reasoning, Tool-Use) ( #38826 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Signed-off-by: Luciano Martins <lucianomartins@google.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Isotr0py <2037008807@qq.com >
2026-04-02 11:13:28 -07:00
+2
104605cbf2
Remove deprecated reasoning_content message field(part-2) ( #37480 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com >
Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
Signed-off-by: yewentao256 <zhyanwentao@126.com >
Signed-off-by: Philip Ottesen <phiott256@gmail.com >
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai >
Signed-off-by: Michael Goin <mgoin64@gmail.com >
Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai >
Signed-off-by: Andy Lo <andy@mistral.ai >
Signed-off-by: Thillai Chithambaram <thillaichithambaram.a@gmail.com >
Signed-off-by: sihao.li <sihao.li@intel.com >
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Co-authored-by: JartX <sagformas@epdcenter.es >
Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com >
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com >
Co-authored-by: Philip Ottesen <phiott256@gmail.com >
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com >
Co-authored-by: Andy Lo <andy@mistral.ai >
Co-authored-by: Thillai Chithambaram <79466435+thillai-c@users.noreply.github.com >
Co-authored-by: sihao_li <165983188+1643661061leo@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-19 15:20:08 +00:00