iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
104 Commits
Author SHA1 Message Date
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack 
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 17:10:56 +07:00
MOVZXandHinata 2b544b7ba0 [Mini-VLLM] Cut dead subsystems across serving, connectors, parsers 
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
  moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
  snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
  cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
  ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
  relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 02:19:03 +07:00
MOVZXandHinata 54a396586e [Mini-VLLM] Repair test suite after SM86 cut 
- Restore example connector, test helpers, and dropped patch tests
- Guard MLA metadata import in the example connector
- Point the PLE test at common.ops.ple after upstream #60021 move
- Cut tests for removed features: tools, DRY, epd proxy, STT batch,
  diffusion scheduler, INC/Quark PLE cases, mamba-2 mixer

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 01:06:30 +07:00
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
Afonso RibeiroandFlora Feng 5c1c903501 [Perf][Parser] Stream Qwen3 tool argument values incrementally (12 KB call: 2.1 s to 11 ms) (#62027) 
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-17 18:49:06 +00:00
Afonso RibeiroandFlora Feng b088eba944 [Perf][Parser] Stream MiniMax M2 tool argument values incrementally (16 KB call: 5.9 s to 26 ms) (#62035) 
Signed-off-by: Afonso Ribeiro <afonso.diamantino2@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-16 22:46:02 +00:00
Woosuk KwonBugen ZhaoYoni GozlanneilkgCursorClaude Opus 5.5
 		153a17166e [Frontend] Add the Beam reasoning and tool parsers (#62052) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Co-authored-by: neilkg <33635204+neilkg@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 14:08:29 -07:00
alberto ae8f41238c [Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts (#61857) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-10-16 12:54:42 -04:00
Maroon AyoubClaudeRobert Shaw
 		4ac1f8d2ac [Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends (#61574) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 12:53:41 +08:00
Ben BrowningandFlora Feng 4c5ea86f53 [Bugfix][Parser] Stream Gemma4 string tool arguments incrementally (#61494) 
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-14 21:25:41 +00:00
Abhay JoshiGeminiArtem Perevedentsev
 		3f466c852c [Bugfix][Parser] Recover genuine tool calls after unclosed Qwen3 <function= opener in prose (#59928) 
Signed-off-by: Abhay Joshi <105213625+abhayjoshi201@users.noreply.github.com>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Gemini <noreply@google.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
 2026-10-14 13:01:25 +03:00
Yoni GozlanandBugen Zhao 69d00bc264 [Bugfix] Accept whole-number floats for integer tool arguments in the response parser (#61297) 
Signed-off-by: Yoni Gozlan <74535834+yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-13 12:55:20 +08:00
Maroon AyoubandFlora Feng 44f0ddfd7b [Bugfix][Parser] Stop re-parsing the whole GLM tool argument per chunk (#61253) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-12 21:28:35 +00:00
tc-mbandCyrus Leung c85616efc6 [Model] Support MiniCPM-V 4.7 (#58674) 
Signed-off-by: tc-mb <tianchi_cai@icloud.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-11 10:21:10 +00:00
simon-veitner-redhatClaude Opus 5.5Misha GoinYifan Zong
 		c23ca06b6d [Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar (#59627) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com>
 2026-10-08 19:38:33 +00:00
Dhairyashil R Gandsfeng33 d6fe5dca68 [Bugfix][Parser] Emit buffered post-reasoning text when no tool parser is configured (#58911) 
Signed-off-by: Dhairyashil R G <18738735+dhairyashilRG@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-10-07 23:21:51 +00:00
Bugen ZhaoandClaude Opus 5.5 5145e35e50 [Frontend] Port the MiniMax M3 tool parser to the parser engine (#59743) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
lzhan011andFlora Feng a3e0243b1c [Bugfix][Parser] Mistral pre-v11: don't fail requests on unexpected tool call JSON (#54844) 
Signed-off-by: lzhan011 <35493221+lzhan011@users.noreply.github.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 03:37:30 +00:00
Mai Hoàng ĐứcandFlora Feng 5cd8a2626b [Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls (#60036) 
Signed-off-by: Mazukiri <maihoangduc04@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 00:06:59 +00:00
Yoni GozlanCursorBugen ZhaoClaude Opus 5.5
 		9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 02:33:00 +00:00
Flora Feng a5105dba05 [Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 22:18:37 +00:00
Yoni Gozlan 2eaa3bc5ac [Frontend] Enrich chat_parsing streaming events (#58603) 
Give serving adapters the delimiter spans and opener captures they need,
and keep streaming after a region that fails to parse.

Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Fix mypy index types in the event-stream test.
 2026-10-01 11:13:13 +08:00
Yoni GozlanYoni GozlanCursormergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		73a5831127 [Frontend] Port chat_parsing core from Transformers (#58602) 
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-10-01 00:21:02 +08:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
yzong-rh 1d331f8341 [Bugfix] Suppress HarmonyError Unexpected token while expecting start token 200006 (#59254) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-09-29 16:40:15 -04:00
Yixin DongCodexyuchuanBugen ZhaoClaude Opus 5.5
 		dfc8e0f3e2 [Frontend] Support strict MiMo-V2.6 tool calling (#58019) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 12:47:47 +00:00
Václav ČadekandCursor Agent 6cbbea8721 [Bugfix] Hoist $defs/definitions in Cohere parser tool schema composition (#49602) 
Signed-off-by: Václav Čadek <vaclavcadek@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2026-09-29 07:11:56 +00:00
Shinichi Hemmi 17cc3cd80e [Model] Engine based plamo3 parser (#56575) 
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp>
 2026-09-29 05:17:42 +00:00
Artem PerevedentsevYufeng HepablopupohubuntClaude Sonnet 5
 		2b9b55c7f1 [Bugfix] Support repsonse_format + tool_choice=auto (#56086) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com>
Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-27 21:17:34 +04:00
Wentao Ye bf93e1f92c [Refactor] Remove dead tests utils (#58803) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-09-26 09:16:53 -04:00
Baljinder HothiandFlora Feng fdcce47e9b [Bugfix][Frontend] Fix Inkling tool name leaking into content after reasoning (#58792) 
Signed-off-by: Baljinder Hothi <baljinder.hothi@cohere.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-26 00:38:52 +00:00
Yousafandsfeng33 83db8c8390 [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) 
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-09-25 20:43:08 +00:00
Samyabrata MajiandFlora Feng 064bc747ab [Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626) 
Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-25 00:12:29 +00:00
Nikhil KulkarniClaude Opus 4.8Chauncey
 		15ed1262e7 [Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine (#49648) 
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 10:37:40 +00:00
lazymioClaude Opus 5Bugen Zhao
 		39e33db7f3 [Frontend] Fix the parsing of missing string= in DeepSeek V4 (#56271) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
 2026-09-18 05:11:24 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
dev 8be5205abb [Parser] Fix: correct parser frontend handling of reasoning end and boundary tokens (#56635) 
Signed-off-by: DEV TYAGI <devtyagi3909@gmail.com>
 2026-09-16 01:17:22 -07:00
yzong-rh e6960af33b [Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200) 2026-09-15 09:48:49 -04:00
Flora Feng 24bfbd5d4d [Refactor] Derive is_reasoning_end from the engine grammar (#56200) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-15 10:07:58 +00:00
Walter Beller-MoralesandFlora Feng 756794a9a7 [Frontend] create unified Cohere parser (#56392) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-12 04:04:46 -04:00
Roger WangKimi Codezjy0516Codex
 		9b959b8657 [Model] DeepSeek-V4.1-Flash Model Definitions (#56228) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-10 20:16:26 +08:00
zexplorerhjCodexmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		fe26c705f7 [Model] Add Bailing V3 VL support (#55921) 
Signed-off-by: zexplorerhj <zhjoneson@163.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-10 02:46:56 -07:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 01:15:06 -07:00
Flora Feng 9e25706560 [Bugfix] Tolerate misspelled DSML tool_calls wrapper (#56141) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-10 15:04:29 +08:00
Mikhail PodvitskiiandFlora Feng 65f3fca568 [Bugfix][Parser] Seed-OSS turn-boundary tokens + boundary-fallback tests (#54264) 
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-09 23:29:30 +00:00
Flora Feng d98c8c030b [Bugfix] Parse DSML tool calls when the model omits the tool_calls wrapper (#55954) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-08 17:38:04 -07:00
Flora FengandWentao Ye 29f46ac475 [Perf] Eliminate full-history reasoning scans for structured outputs (#55223) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-08 14:07:16 -07:00
Flora Feng 300f688321 [Bugfix] Implicitly close DeepSeek DSML parameters (#54838) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-02 14:13:36 +08:00
Suvrakamal DasClaudeBen Browning
 		e7cf4730d6 [Bugfix] Drop incomplete tool-call markup in non-streaming to match streaming (#47562) 
Signed-off-by: Suvrakamal Das <jyb2084.usa@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-01 18:11:36 +00:00