iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
46 Commits
Author SHA1 Message Date
MOVZXandHinata 2b544b7ba0 [Mini-VLLM] Cut dead subsystems across serving, connectors, parsers 
- Remove P/D transfer connectors: nixl, nixl_chat, mooncake x2, hf3fs,
  moriio, hisparse, example and hidden-states connectors
- Reduce weight_transfer to the base API used by model loading
- Remove kv_offload tiering, simple_kv_offload, ec cpu/example,
  snapshot command, engine snapshot server
- Remove transformers model fallback, dsv4 tokenizers, dead platforms
  cpu/tpu/xpu, flashmla third_party, dsa, megamoe/flydsl, dead attn
  ops, dead layers (short_conv, fused_embed_norm, indexer_topk,
  relu2 gemma_ar, lightning_attn, minimax_rms_norm), gemm_rs_ar
- Keep MoE LoRA op (imported eagerly by punica_gpu); drop fp8 variant
- Rebuild parser hubs around Qwen3 adapters only (36 dead adapters)
- Remove dead tests with the code they covered

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 02:19:03 +07:00
alberto ae8f41238c [Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts (#61857) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-10-16 12:54:42 -04:00
Maroon AyoubClaudeRobert Shaw
 		4ac1f8d2ac [Bugfix][Structured Output] Mask grammar-invalid implicit reasoning ends (#61574) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-10-16 12:53:41 +08:00
Ben BrowningandFlora Feng 4c5ea86f53 [Bugfix][Parser] Stream Gemma4 string tool arguments incrementally (#61494) 
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-14 21:25:41 +00:00
Maroon AyoubandFlora Feng 44f0ddfd7b [Bugfix][Parser] Stop re-parsing the whole GLM tool argument per chunk (#61253) 
Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-12 21:28:35 +00:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
simon-veitner-redhatClaude Opus 5.5Misha GoinYifan Zong
 		c23ca06b6d [Bugfix][Structured Output] Feed the token that implicitly ends reasoning to the grammar (#59627) 
Signed-off-by: Simon Veitner <sveitner@redhat.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: Yifan Zong <263738792+yzong-rh@users.noreply.github.com>
 2026-10-08 19:38:33 +00:00
Bugen ZhaoandClaude Opus 5.5 5145e35e50 [Frontend] Port the MiniMax M3 tool parser to the parser engine (#59743) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
Flora Feng a5105dba05 [Frontend] Port Step-3.5 parsers to the streaming parser engine (#59321) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-10-01 22:18:37 +00:00
Yixin DongCodexyuchuanBugen ZhaoClaude Opus 5.5
 		dfc8e0f3e2 [Frontend] Support strict MiMo-V2.6 tool calling (#58019) 
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Ubospica <ubospica@gmail.com>
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-29 12:47:47 +00:00
Shinichi Hemmi 17cc3cd80e [Model] Engine based plamo3 parser (#56575) 
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp>
 2026-09-29 05:17:42 +00:00
Yousafandsfeng33 83db8c8390 [Feature][Frontend] Add granite_thinking_parser reasoning parser for Granite 4.2 (#55957) 
Signed-off-by: Yousaf shah <yousaf.shah@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-09-25 20:43:08 +00:00
Nikhil KulkarniClaude Opus 4.8Chauncey
 		15ed1262e7 [Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine (#49648) 
Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-09-23 10:37:40 +00:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
dev 8be5205abb [Parser] Fix: correct parser frontend handling of reasoning end and boundary tokens (#56635) 
Signed-off-by: DEV TYAGI <devtyagi3909@gmail.com>
 2026-09-16 01:17:22 -07:00
yzong-rh e6960af33b [Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200) 2026-09-15 09:48:49 -04:00
Flora Feng 24bfbd5d4d [Refactor] Derive is_reasoning_end from the engine grammar (#56200) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-15 10:07:58 +00:00
Bugen ZhaoandIsotr0py b47b01cf33 [Model][Frontend] Support DeepSeek-V4.1-Flash in Rust and Python frontends (#56208) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
 2026-09-10 01:15:06 -07:00
Flora Feng 9e25706560 [Bugfix] Tolerate misspelled DSML tool_calls wrapper (#56141) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-10 15:04:29 +08:00
Flora FengandWentao Ye 29f46ac475 [Perf] Eliminate full-history reasoning scans for structured outputs (#55223) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-08 14:07:16 -07:00
Suvrakamal DasClaudeBen Browning
 		e7cf4730d6 [Bugfix] Drop incomplete tool-call markup in non-streaming to match streaming (#47562) 
Signed-off-by: Suvrakamal Das <jyb2084.usa@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-01 18:11:36 +00:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-31 10:13:50 +00:00
Mikhail Podvitskii 3a9bfc209f [Bugfix][Parser] Scope reasoning-end detection to the current turn via turn-boundary tokens (#54089) 
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
 2026-08-27 16:57:32 -04:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ac2ae8798c [Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-15 06:51:14 -07:00
Jason 0820125ae9 [Bugfix][Parser] Emit REASONING_END for Inkling tool calls that follow no thinking block (#50528) 
Signed-off-by: Jason Yao <wsyjh8@gmail.com>
 2026-08-09 21:04:53 -04:00
cherry77-cloudBen BrowningClaude Opus 4.8
 		75231eff2f [Bugfix][Parser] Prevent Inkling block-end leakage with tools (#51391) 
Signed-off-by: cherry77-cloud <1615405@qq.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-08 12:34:32 -04:00
Bugen Zhao 629a938a92 [Frontend] Preserve bare Inkling text in Python and Rust parsers (#50403) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-07-30 12:25:10 -07:00
Julien Denize 1a20d23dab [PARSER][Mistral] unified engine-based parser for reasoning and tool calls (#48947) 2026-07-30 15:38:00 +02:00
Ben Browningandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 3de4b2bf3c [Bugfix][Parser] Fix special tokens (EOS/BOS) leaking into reasoning content (#48748) 
Signed-off-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-22 16:10:55 -04:00
Woosuk KwonBugen ZhaoGiancarlo DelfinIsotr0pyIsotr0pyJee Jee LiRoger WangYifan QiaoClaude Fable 5OpenAI Codex
 		6570c9800c [Model] Add Inkling model support [1/N] (#48799) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-15 23:40:07 -07:00
albertoandChauncey 5715fde12c [Feature][Parser] Support include_reasoning param for non-Harmony models (#44301) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-07-10 15:34:02 +08:00
Ben Browning fb5291b35b [Frontend] [Parser] Port DeepSeek V4 to streaming parser engine framework (#45877) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-07-03 20:55:23 -04:00
Ben BrowningandClaude Opus 4.6 25671cb520 [Parser][Bugfix] Ensure tool call or other special tokens don't leak in non-streaming tool parsing (#46875) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-06-30 13:46:53 -04:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 2bc20e8aba [Frontend] Add Streaming Parser Engine and new Kimi k2.5/k2.6/k2.7 Parser (#46610) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-30 07:53:17 +00:00
Maxwill Lin cd347298e8 [Frontend] Port seed_oss to the streaming parser engine as a Qwen3 subclass (#46314) 
Signed-off-by: EazyReal <8047065+EazyReal@users.noreply.github.com>
 2026-06-24 20:08:42 -04:00
Chauncey 84c2f9f0fb [Frontend] Fix Kimi K2 tool call IDs for required tool choice (#46344) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-06-24 19:59:40 +00:00
Rui Yinandabinggo 8db12169a4 fix: stream Qwen3 tool call string arguments (#46351) 
Signed-off-by: Rui Yin <2260891073@qq.com>
Co-authored-by: abinggo <107740309+abinggo@users.noreply.github.com>
 2026-06-23 10:26:37 +08:00
Ben Browning 859e4d436b [Bugfix][Parser] Fix U+FFFD leak at reasoning-to-content transition in engine parsers (#46159) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-06-19 22:09:28 +00:00
Ben BrowningandFlora Feng 7f616c327d [Bugfix] [Parser] Fix empty tool block silently dropping subsequent content (#46091) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-06-18 23:17:18 +00:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6c379b9e54 [Frontend] Add Streaming Parser Engine and new GLM4.7/GLM5.1/GLM5.2 Parser (#45915) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-19 00:42:10 +08:00
Nikhilesh Chhetri 3c6084bb0d [Bugfix][Gemma4] Pre-initialise streaming reasoning state when prompt ends inside an open <|channel> (fixes #45834) (#45852) 
Signed-off-by: nikhilesh-csa <nchhetri@csa1.com>
 2026-06-17 06:16:02 -04:00
Chauncey f00e163f35 [Frontend] Add Streaming Parser Engine and new MinimaxM2 Parser (#45701) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-06-16 13:38:17 -04:00
Ben Browning f3858d5422 [Frontend] [Parser] Migrate Nemotron V3 to streaming parser engine (#45755) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-06-16 05:31:21 +00:00
Ben BrowningandFlora Feng 76a373eff4 [Frontend] Replace legacy Gemma4 parsers with engine-based implementation (#45588) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-06-15 21:34:07 +00:00
Ben BrowningandFlora Feng c4a3f9d137 [Frontend] Add Streaming Parser Engine and new Qwen3 Parser (#45413) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-06-15 11:59:05 +08:00