iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,342 Commits 3 Branches 0 Tags
sm86
Commit Graph
59 Commits
Author SHA1 Message Date
MOVZXandHinata 2066d3143f [Mini-VLLM] Cut non-OpenAI surfaces, dead kernels, Mistral stack 
- Remove Anthropic, Cohere, SageMaker, decisions, structured-decisions,
  rl, render-launcher and batch entrypoints; keep scale-out token routes
- Drop the Swagger static bundle and the /docs, /redoc routes
- Delete device-keyed tuning configs for fused_moe, quant utils, mamba
  select-state and helion; no sm86 entry ever existed for any of them
- Remove ten unused rope variants plus the fbgemm_fp8 and fp_quant
  quantizers and their config, metrics and loader plumbing
- Cut the CPU device build: csrc/cpu, cpu_extension.cmake, cpu
  requirements and the setup.py cpu target path
- Delete 29 environment variables nothing reads and 14 chat-template
  fallbacks for models this fork no longer registers
- Remove nine dead CUDA ops (deepseek_v4, kimi_k3, minimax, muse_spark,
  dsv3 gemm) with their kernels, bindings, wrappers and CMake entries
- Delete the connector test suites that targeted removed connectors
- Cut the Mistral tokenizer and parser stack, its tests, and the
  mistral_common and ijson dependency pins
- Restore the Qwen4Exp registry rows and nvidia/mtp.py that an earlier
  cut deleted while keeping the model tree

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 17:10:56 +07:00
alberto ae8f41238c [Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts (#61857) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
 2026-10-16 12:54:42 -04:00
Bugen ZhaoandClaude Opus 5.5 8fa5ae844a [Frontend] Let an unset --tool-strict-level fall back to the tool parser's default (#61328) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 15:48:38 +08:00
Shijin ZhangandChauncey 83773ecdf3 [Frontend] Support CustomToolCall in the non-Harmony Responses API (#57322) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-10-12 17:22:55 +08:00
wang.yuqi c1692c85f1 [Misc] mv chat_utils.py to renderers. (#60522) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-10-10 06:50:34 +00:00
Junyi Yaoandsfeng33 fdfc171a74 [Bugfix][ToolParser] Stream every tool call when one delta completes several (#60354) 
Signed-off-by: Junyi Yao <j.yao@wustl.edu>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-10-08 01:07:35 +00:00
Dhairyashil R Gandsfeng33 d6fe5dca68 [Bugfix][Parser] Emit buffered post-reasoning text when no tool parser is configured (#58911) 
Signed-off-by: Dhairyashil R G <18738735+dhairyashilRG@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-10-07 23:21:51 +00:00
Mai Hoàng ĐứcandFlora Feng 5cd8a2626b [Bugfix][Structured Output] Cap JSON schema nesting to prevent 500s and API-server stalls (#60036) 
Signed-off-by: Mazukiri <maihoangduc04@gmail.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-06 00:06:59 +00:00
RomeJayandFlora Feng 2e06395fd3 [Bugfix][Frontend] Build tool-call grammars from the tools the prompt renders (#59879) 
Signed-off-by: Rongjie Jiang <j1870831863@outlook.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-05 22:01:03 +00:00
Shijin ZhangandFlora Feng d1a974c140 [Frontend] Constrain non-strict GLM-4.7 tool calls with a shallow structural tag (#56403) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-10-02 14:31:53 -04:00
Yoni GozlanCursorBugen ZhaoClaude Opus 5.5
 		9bd993564f [Frontend] Parse tool calls and reasoning from checkpoint response templates (#58604) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-10-02 02:33:00 +00:00
Ostring24andFlora Feng 4e7af3b0de [Bugfix][Frontend] Enforce parallel_tool_calls=false in the required-tool grammar (#50502) 
Signed-off-by: ostring <ostring@qq.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-09-29 21:33:28 +00:00
Shinichi Hemmi 17cc3cd80e [Model] Engine based plamo3 parser (#56575) 
Signed-off-by: Shinichi Hemmi <shemmi@preferred.jp>
 2026-09-29 05:17:42 +00:00
Artem PerevedentsevYufeng HepablopupohubuntClaude Sonnet 5
 		2b9b55c7f1 [Bugfix] Support repsonse_format + tool_choice=auto (#56086) 
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: pablopupo <145598901+pablopupo@users.noreply.github.com>
Co-authored-by: hubunt <150658615+hubunt@users.noreply.github.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-27 21:17:34 +04:00
lazymioClaude Opus 5Bugen Zhao
 		dee37d8911 [Frontend] Add --tool-strict-level for server-side control for structural tag activation (#56268) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: lazymio <mio@lazym.io>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-09-18 04:32:39 +00:00
Flora Feng e0705731be [Refactor] Remove unused interface methods (#57194) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-09-16 21:00:48 +00:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
yzong-rh e6960af33b [Refactor] StructuredOutputManager x Speculative Decoding Refactor (#48200) 2026-09-15 09:48:49 -04:00
Flora FengandWentao Ye 29f46ac475 [Perf] Eliminate full-history reasoning scans for structured outputs (#55223) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
 2026-09-08 14:07:16 -07:00
Suvrakamal DasClaudeBen Browning
 		e7cf4730d6 [Bugfix] Drop incomplete tool-call markup in non-streaming to match streaming (#47562) 
Signed-off-by: Suvrakamal Das <jyb2084.usa@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-09-01 18:11:36 +00:00
wang.yuqiandCyrus Leung eeb549a74d [Frontend] Move engine/protocol.py out openai folder (#54492) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-08-31 10:13:50 +00:00
Chaunceyandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> ac2ae8798c [Frontend] Support count_reasoning_tokens in the Streaming Parser Engine (#45802) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-15 06:51:14 -07:00
Chauncey 6c91de3689 [Bugfix][Parser] Forward model_config to nested reasoning parsers (#50642) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-08-01 13:07:34 +08:00
akii96andBen Browning 550218b136 [Bugfix][Frontend] Flush engine reasoning parser at engine-reasoning → tool streaming boundary (#47606) 
Signed-off-by: Aakif Nawaz <aakif.nawaz@amd.com>
Co-authored-by: Ben Browning <56071+bbrowning@users.noreply.github.com>
 2026-07-13 14:06:10 -04:00
albertoandChauncey 5715fde12c [Feature][Parser] Support include_reasoning param for non-Harmony models (#44301) 
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2026-07-10 15:34:02 +08:00
Ben BrowningandClaude Opus 4.6 25671cb520 [Parser][Bugfix] Ensure tool call or other special tokens don't leak in non-streaming tool parsing (#46875) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-06-30 13:46:53 -04:00
Chauncey 84c2f9f0fb [Frontend] Fix Kimi K2 tool call IDs for required tool choice (#46344) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-06-24 19:59:40 +00:00
Ben Browning 859e4d436b [Bugfix][Parser] Fix U+FFFD leak at reasoning-to-content transition in engine parsers (#46159) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2026-06-19 22:09:28 +00:00
Nikhilesh Chhetri 3c6084bb0d [Bugfix][Gemma4] Pre-initialise streaming reasoning state when prompt ends inside an open <|channel> (fixes #45834) (#45852) 
Signed-off-by: nikhilesh-csa <nchhetri@csa1.com>
 2026-06-17 06:16:02 -04:00
Ben BrowningandFlora Feng c4a3f9d137 [Frontend] Add Streaming Parser Engine and new Qwen3 Parser (#45413) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
 2026-06-15 11:59:05 +08:00
Chauncey 3b8fc3fe6d [Frontend] Support strict mode for tool calling with ResponsesAPI (#45396) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-06-12 10:59:59 -04:00
Chaunceycjackalmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2043258dec [Frontend] Support strict mode for tool calling (#45003) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: cjackal <44624812+cjackal@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-12 07:51:48 +00:00
yzong-rh f712fd0d7d [Refactor] Chat Completions Harmony Refactor, non-streaming path. (#45171) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-06-11 21:18:30 +00:00
Flora Feng 3a04061701 [Refactor][Parser] Unify Response API to use parser.parse() like Chat Completion API (#45190) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-11 04:37:51 +00:00
yzong-rhandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 6ec7dcd641 [Frontend][Metrics] Add vllm:tool_call_parser_invocations_total Prometheus metric (#44448) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-10 10:29:11 -04:00
Andrii SkliarandAndrii Skliar 4882fd7632 [Bugfix][Reasoning] Nemotron V3: surface reasoning as content when thinking is unterminated (#39091) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
 2026-06-10 05:58:19 -07:00
Flora Feng d82ac00923 [Refactor][Mistral] Extract parsing logic into MistralParser (#44596) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-10 00:12:23 +00:00
hoobnnandsfeng33 2b237c7a41 [Bugfix] Honor tool_choice="none" in Chat Completions streaming (#42752) 
Signed-off-by: hoobnn <111053672+hoobnn@users.noreply.github.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-06-03 13:27:45 -07:00
Flora Feng 209709a8c1 [Bugfix] Fix unstreamed tool call args dropped in Responses API streaming (#44348) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-03 03:19:08 -07:00
Flora Feng 478b49ddec [Refactor] Remove dead code from parser infrastructure (#44279) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-02 12:08:27 -04:00
Flora Feng 68dafcca75 [Refactor] Unify reasoning + tool-call parsing behind Parser.parse() (#44267) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-02 15:11:42 +08:00
Flora Feng 9affc17a05 [Refactor] Move unstreamed tool-arg flush from serving layer to parser (#44017) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-06-02 10:37:43 +08:00
Flora Feng b32fe416ea [Bugfix] Fix reasoning dropped on streaming boundary deltas (#42691) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-23 16:18:30 -07:00
Flora Feng 8eb401134e [Refactor] Consolidate required/named tool_choice streaming into DelegatingParser (#41876) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-05-07 09:50:59 -07:00
xy3andsfeng33 50acdc5b5c Fix Qwen3 streaming content routing (#40820) 
Signed-off-by: xy3 <120182408@qq.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Co-authored-by: sfeng33 <4florafeng@gmail.com>
 2026-05-06 17:22:01 -04:00
jackQwertyJackchaunceyjiang
 		6467213a9f fix(openai): tolerate empty content in forced tool choice (#40148) 
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-05-06 07:16:03 -07:00
Chauncey 3885d340a4 [Frontend]Responses API supports Tool/Function calling with streaming with named tool/function (#41110) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-29 09:11:27 +00:00
Chauncey 146f44b77d [Frontend]Responses API supports Tool/Function calling with streaming with required (#40700) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2026-04-27 19:36:58 -07:00
Flora Feng db8a6d66bf [Refactor][Parser] Migrate chat completion auto-tool/reasoning/plain streaming to parse_delta (#39446) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-14 04:39:45 +00:00
Flora Feng 14cb86c187 [Refactor][Parser] Simplify parse_delta (#39728) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-04-13 21:02:13 +00:00