iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,332 Commits 3 Branches 0 Tags
81074e94a3d4c8e792d7dc708e9540c29213bc6f
Commit Graph
63 Commits
Author SHA1 Message Date
MOVZXandHinata 81074e94a3 [Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests 
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-19 00:20:48 +07:00
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
KevinClaude Opus 5.5Kevin H. Luu
 		d12ea8e309 [Bugfix][MRV2] Keep input_ids on later PP stages for raw-token models (#61016) 
Signed-off-by: booth-algo <kevinlauofficial01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-15 12:54:21 -07:00
haotiaa 6e6e86078a [Bugfix][Platform] Don't resolve current_platform during import vllm (#60766) 
Signed-off-by: Haotian An <haotiaa@amazon.com>
 2026-10-12 17:05:34 +00:00
shaohuaxiCodexCyrus Leung
 		d4e92ccc37 [Bugfix][Frontend] Honor endpoint plugin route overrides (#58839) 
Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-12 03:58:15 +00:00
Harry MellorandClaude Opus 5.5 2c99ee9333 [Chore] Remove vendored code made redundant by Transformers 5.16.1 (#60623) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 09:29:03 +00:00
Harry MellorandClaude Opus 5.5 fc2c801ad9 Disable mypy arg-type and assignment checks in tests (#59428) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-30 15:18:50 +00:00
Martin HickeyHarry MellorClaude Opus 5.5
 		5dd693122c [MyPy][2/N] Fix mypy errors in small tests/ dirs (#51043) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 21:27:45 +01:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
wang.yuqi 735daf8ef6 [Frontend] Move cli_args.py and dp_supervisor.py out openai folder (#53659) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-25 07:02:32 +00:00
wang.yuqi eac636a7fa [Frontend] Move api_server.py out openai folder (#52131) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-08-19 09:43:46 +00:00
Isotr0py 073c510c91 [Migration] Migrate bitsandbytes support to OOT plugin (#43529) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 10:27:31 +08:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
Harry MellorandAndreas Karatzas ca0defa343 Make bare hugging_face imports forbidden (#49726) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-25 04:18:39 -07:00
Flora Feng fca252d59e [CI][Bugfix] Reduce max_model_len in OOT embedding test to fix KV-cache OOM on small GPUs (#49351) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2026-07-21 15:00:06 -04:00
aoshen02aoshen02Codex
 		d9aa35161d Update BGE-M3 token expectations for leading spaces (#49269) 
Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-07-21 03:49:17 +00:00
Martin Hickey f7fc0ca993 [Frontend] Add endpoint plugins framework (#47454) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
 2026-07-08 10:00:41 +08:00
Nick Hill e9993a52aa [BugFix][CI] Fix scheduler plugin test (#45897) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-06-17 06:30:49 +00:00
xx-thomas b00e76ff72 [Misc][Model] add io processor for query/document embeddings from ColBERT (jinaai/jina-colbert-v2) (#45210) 
Signed-off-by: thomas <thomas.varghese@columbia.edu>
 2026-06-16 01:32:32 +00:00
Isotr0py 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 12:02:21 -07:00
wang.yuqi 996222f4bf [CI] Reorganize entrypoints CI (#44947) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2026-06-09 00:46:11 -07:00
Stefano Castagnetta efb4cdf2b8 [CI/Build] Skip Prithvi/Terratorch model-registry tests when terratorch is missing (#41389) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-30 12:47:55 -07:00
Stefano Castagnetta 10558f5f46 [CI/Build] Skip terratorch + torchgeo while PyPI has lightning quarantined (#41377) 
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
 2026-04-30 07:59:07 -07:00
Cyrus Leung ba2f0acc2d [Misc] Reorganize inputs (#35182) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-03-25 10:22:54 -07:00
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Cyrus Leung
 		1b6cb920e6 [Deprecate] Deprecate pooling multi task support. (#37956) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-03-24 14:07:47 +00:00
Isotr0py a836524d20 [Chore] Replace all base64 usages with faster pybase64 package (#37290) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-03-17 14:44:19 +00:00
Augusto Yao 9c7cab5ebb [Feature]: Support for multiple embedding types in a single inference call (#35829) 
Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
 2026-03-17 17:05:42 +08:00
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Harry Mellor
 		dcf8862fd4 [Examples][1/n] Resettle basic examples. (#35579) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-03-08 20:22:53 -07:00
Christian Pinto 2f2212e6cc Split generic IO Processor plugins tests from Terratorch specific ones (#35756) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2026-03-05 00:01:03 +08:00
Augusto YaoandCyrus Leung 8e75d88554 add io_process_plugin for sparse embedding (#34214) 
Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
Signed-off-by: Augusto Yao <augusto.yjh@antgroup.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2026-02-28 09:16:37 +00:00
Christian Pinto 6930becd45 (bugfix): Fixed encode in LLM entrypoint for IOProcessr plugin prompts (#34618) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2026-02-16 07:33:55 -08:00
Christian PintoandCyrus Leung 342a7cda2d [Misc] Update tests and examples for Prithvi/Terratorch models (#34416) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-02-13 23:03:51 -08:00
Cyrus Leung c9a1923bb4 [Plugin] Simplify IO Processor Plugin interface (#34236) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-10 19:47:39 -08:00
Lucas WilkinsonandLuka Govedič 6cdf015c3c [Misc] Fix Current vLLM config is not set. warnings, assert to avoid issues in the future (#31747) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
 2026-01-08 15:20:49 -08:00
wang.yuqi 62de4f4257 [Frontend] Resettle pooling entrypoints (#29634) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-01 15:30:43 +08:00
wang.yuqigemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Christian Pinto
 		3fa2c12185 [Frontend][4/N] Improve all pooling task | Add plugin pooling task (#26973) 
Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Christian Pinto <christian.pinto@ibm.com>
 2025-10-23 14:46:18 +00:00
Cyrus Leung bfb240cc49 [CI/Build] Fix Prithvi plugin test (#27393) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-23 07:30:44 +00:00
Russell BryantandDarkLight1337 58fab50d82 [Frontend] Require flag for loading text and image embeds (#27204) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-22 15:52:02 +00:00
Tova Movshovitz 83e760c57d [V1][Metrics][Plugin] Add plugin support for custom StatLoggerBase implementations (#22456) 
Signed-off-by: tovam <tovam@pliops.com>
 2025-10-18 15:12:46 -07:00
wang.yuqi f54f85129e [Model][2/N] Improve all pooling task | Support multi-vector retrieval (#25370) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-10-15 11:14:41 +00:00
Cyrus Leung 1e4ecca1d0 [V0 Deprecation] Remove VLLM_USE_V1 from tests (#26341) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-07 15:42:31 +00:00
Harry Mellor d6953beb91 Convert formatting to use ruff instead of yapf + isort (#26247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-05 07:06:22 -07:00
Woosuk Kwon 52c2a8d4ad [V0 Deprecation] Remove LLMEngine (#25033) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-20 17:56:30 -07:00
Christian PintoandCyrus Leung 9cd76b71ab [Misc] Terratorch related fixes (#24337) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-08 06:40:26 -07:00
mgazzChristian PintoCyrus Leung
 		51d5e9be7d [Core][Model] Terratorch backend integration (#23513) 
Signed-off-by: Michele Gazzetti <michele.gazzetti1@ibm.com>
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: Christian Pinto <christian.pinto@ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-04 00:22:41 -07:00
Christian Pinto cf91a89dd2 [docs][misc] IOProcessor plugins fixes (#24046) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2025-09-01 09:17:41 -07:00
Christian PintoandMax de Bayser 1cb39dbcdd [Misc] IO Processor plugins for pooling models (#22820) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
 2025-08-31 23:07:12 -07:00
Michael Goin 35509fc5be [Bugfix] Remove faulty test for oot attention backend (#22286) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-08-06 00:05:40 +00:00
Chendi.Xue 7e8977fcd4 [custom_op][vllm-plugin] update custom_op class to use op_registry (#19164) 
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
 2025-06-20 07:44:56 -07:00