iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
139 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
FrankClaude Opus 5Harry MellorKevin H. Luu
 		3723a1ea96 [Bugfix][Transformers backend] Fix CUDA graph capture for dynamic-rope models (#58064) 
Signed-off-by: YongfengX <erkang040804@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
 2026-10-14 05:56:54 -07:00
Luca MotzClaude Opus 5.5CodexHarry Mellor
 		698e79f3bd [Model] Kolibri 1 via the Transformers modeling backend (#60827) 
Signed-off-by: Luca Motz <luca.motz@icloud.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-13 22:17:52 +01:00
TaimysTaimysHarry Mellor
 		c068db0e1d [Bugfix][Compile] Replace Transformers AOT caching with graph caching (#58918) 
Signed-off-by: fangyao fan <2281693949@qq.com>
Signed-off-by: Taimys <ffy2281693949@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Taimys <ffy2281693949@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-13 19:17:29 +00:00
Harry MellorandClaude Opus 5 6f11c220b5 [Transformers backend] Fuse MoE routers that score after the top-k (#57410) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-13 08:41:23 +00:00
Harshal JanjaniHarry MellorClaude Opus 5
 		34051ad714 feat: Add video support for the Transformers backend (#57441)
pre-commit / pre-run-check (push) Canceled after 0s
Details
pre-commit / pre-commit (push) Canceled after 0s
Details 
Signed-off-by: Harshal Janjani <harshaljanjani@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-10-05 00:51:15 -07:00
Harry MellorandClaude Opus 5.5 6e4efafc6e [Model] Migrate GPT-NeoX, Phi, Seed-OSS and Jais2 to the Transformers modeling backend (#59701) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:18:35 +00:00
Harry MellorandClaude Opus 5.5 5d20979fc9 [Misc] Remove code paths for Transformers < 5.16.1 (#59762) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-02 11:17:24 +00:00
Farhad Al-AminClaude Opus 5.5Harry Mellor
 		af2a582d5c [Bugfix][Spec Decode] Qualify Transformers backend attention layer names with the model prefix (#59293) 
Signed-off-by: Farhad Al-Amin <99006654+Legend-2727@users.noreply.github.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-10-02 11:21:20 +01:00
Michał GanczarenkoClaude Sonnet 5Harry Mellor
 		df5668e5c9 [XPU] Dispatch nn.LayerNorm to fused SYCL kernel via CustomOp (#57172) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-30 12:19:33 +00:00
Thomas OrtnerandHarry Mellor 8b84e15066 [transformer] RMSNorm matching for alternative rsqrt (#54461) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-24 16:33:06 +00:00
Harry MellorandClaude Opus 5.5 d881812500 [Chore] Use Transformers v5 names and drop redundant processor use_fast (#58550) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 14:14:10 +00:00
Harry MellorandClaude Opus 5 8a5cf54387 [Multimodal] Type dummy options per modality (#57576) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-18 17:29:11 +01:00
Taneem IbrahimandHarry Mellor e960ead0aa [Mypy] Fix mypy typing for Transformers models (#54320) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-17 12:11:31 -04:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Cyrus Leung 0b9e018fb9 [MM] Further cleanup _apply_hf_processor_main (#53610) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-09-16 10:21:00 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Harry MellorandClaude Opus 5 719284fe15 [Core][Model] Unify XD-RoPE into M-RoPE and derive the channel count (#56078) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-09 09:23:30 -07:00
Thomas OrtnerandHarry Mellor e509d32b5a [Transformers backend] Enable QKV-Fuser for Gemma4 (#55690) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Thomas Ortner <bohnstinglthomas@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-09 14:46:31 +01:00
Jim BanClaudeHarry Mellor
 		c0d8d5413e [Transformers] Generalize merged-column linear fusion (#55301) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-08 23:49:37 +00:00
Thomas ParnellClaudeHarry Mellor
 		414057a3d3 [Bugfix] Apply attention sinks in the Transformers backend (#52156) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-08 13:54:07 +00:00
Thomas OrtnerandHarry Mellor 8ad2076c4a [Transformers backend] Find attention with a fuser and attach vLLM's layer to it (#54941) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-09-04 16:26:01 +00:00
Taneem Ibrahim 758c79e1c5 [Bugfix] Retain vocab embeddings during replacement (#55083) 
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
 2026-09-03 04:17:38 -07:00
Nick Hillandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> a566ea7e8f [Perf] Avoid more h2d copies from non-pinned tensors (#54660) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-01 20:37:09 -07:00
Harry MellorandClaude Opus 5 f1e5fdd7f2 [Transformers backend] Replace vocab embeddings in recursive_replace (#54760) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-01 15:35:24 +00:00
Harry MellorXianbao QIANClaude Opus 5Cyrus Leung
 		48d7132962 [Model] Migrate FlexOlmo, Olmo3 and Hunyuan V1/VL to the Transformers modeling backend (#53615) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Xianbao QIAN <xianbao.qian@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 22:39:05 -07:00
Cyrus Leung ecfa7bb373 [MM] Cache common token sequences (#53560) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-08-24 07:48:10 -07:00
Cyrus LeungandQwen Code ebcd606467 [MM] Simplify prompt updates: replace PromptSeq with list[int] (#53372) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Qwen Code <noreply@alibaba-inc.com>
 2026-08-23 14:50:32 +08:00
Cyrus Leung 0b19ebcacd [MM] Simplify _apply_hf_processor_main (#53275) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-22 01:49:37 -07:00
Cyrus Leung b00f475f09 [MM] Remove text components from ProcessorInputs (#53093) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-08-20 20:42:53 -07:00
Harry Mellor 1fe3a1571a Reduce AutoWeightsLoader kwargs (#53106) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 15:40:11 +00:00
Harry Mellor 2dd17225c6 Fix Transformers modelling backend RMSNormFuser.fuse performance (#52766) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 15:21:23 +00:00
Harry Mellor f0c14b4f77 Fix weight tying (#51665) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 10:31:55 +00:00
Harry Mellor 6e85feb1b7 [3/N] Harden Transformers modelling backend multi-modal path (#51827) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-20 10:24:41 +00:00
Nick Hill deeeae75d0 [MM] Keep more metadata tensors on CPU (#52827) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
 2026-08-18 19:56:33 -07:00
Nick HillandClaude Opus 4.8 615d4cfade [Core] Check for GPU<->CPU syncs during CI (#43107) 
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-08-14 17:42:05 -07:00
Harry Mellor b245d8e73f Apply logit softcapping in Transformers modelling backend (#52173) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-13 16:21:55 +00:00
Thomas Ortner b8baa31a28 Hardware-agnostic model definition via HF transformer backend (1/N) (#49458) 
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
 2026-08-13 11:21:50 +01:00
Andreas KaratzasandOpenAI Codex 1ab2801dde [ROCm] Remove stale SDPA and skinny GEMM workarounds (#50907) 
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-08-11 14:19:16 -04:00
Harry Mellor c65aa2ee6c Bump Transformers version to 5.15.0 (#51668) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-11 09:00:51 -07:00
Harry Mellor be2e274ef5 Fix docs on main (#51773) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 23:52:04 -07:00
Harry Mellor cf8f3a3bb2 [2/N] Harden Transformers modelling backend multi-modal path (#51657) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 12:57:12 +00:00
Harry Mellor 70b84f0bcb Fix Gemma 4 for upcoming Transformers version (#49797) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-10 07:48:26 +00:00
Harry Mellor 45273b8dcb [1/N] Harden Transformers modelling backend multi-modal path (#51408) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-07 17:30:43 +00:00
Harry Mellor e7b8d59460 Fully generalise input embedding handling in Transformers modelling backend (#51247) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-06 16:21:24 +00:00
Anuj BolewarAnuj BolewarHarry Mellor
 		166f4e2dc3 fix: fuse weightless RMSNorms at their declared width (#50867) 
Signed-off-by: Anuj Bolewar <anujbolewar@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Anuj Bolewar <anujbolewar@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-04 16:03:09 +00:00
Michał Ganczarenko 7c40d61eaf [Bugfix] Flatten >2D multimodal embeddings, not just 3D (#50250) 
Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com>
 2026-08-04 13:34:58 +00:00
Harry Mellor 59b2fdfc4e Support MLA properly in the Transformers modeling backend (#48250) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-08-04 13:24:11 +01:00
bnellnmandHarry Mellor 6e311c6e20 [MoE Refactor] Rename FusedMoE to FusedMoEFactory (#44941) 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-31 11:28:18 +01:00
Isotr0py 1180b604f8 [Multimodal] Expose mm hash algothrim selection to cli args (#49686) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-07-31 05:25:19 +00:00