iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
312 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
Rohan PotdarClaude Opus 5.5Micah WilliamsonKevin H. LuuAndreas Karatzas
 		b2456c622b [ROCm][CI] Make ROCm 10.0 the default image; keep ROCm 7.2 as -rocm72 (#58761) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-10-17 21:43:59 -05:00
Ganesh RKevin H. LuuCyrus Leung
 		91ed2859c7 [CPU][Zen] Update zentorch to 2.14.0.1 & enable float16 on Venice and later (#61411) 
Signed-off-by: Ganesh R <ganesh.r@amd.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-10-15 00:23:50 -07:00
Itay EtelisRobert ShawMisha Goin
 		13dbf96093 [Model Loader] Apply EP weight filtering in InstantTensor (#61167) 
Signed-off-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Misha Goin <mgoin64@gmail.com>
 2026-10-14 09:10:10 -07:00
Andrey TalmanAndrey TalmanClaude Opus 5.5
 		30e79c3a76 [Core] Update PyTorch to 2.14.0, torchvision to 0.29.0, triton to 3.8.0 (release channel) (#52183) 
Signed-off-by: Andrey Talman <atalman@users.noreply.github.com>
Signed-off-by: Andrey Talman <atalman@fb.com>
Co-authored-by: Andrey Talman <atalman@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 00:17:59 -07:00
Andrii SkliarAndrii SkliarLuke Alonso
 		6849ca696a [Quantization] Add ModelOpt IQ2_XS, IQ2_XXS, and Q8_0 support via b12x (#59604) 
Signed-off-by: Andrii Skliar <askliar@nvidia.com>
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Andrii Skliar <askliar@nvidia.com>
Co-authored-by: Luke Alonso <lalonso@gmail.com>
 2026-10-13 09:06:33 -07:00
Greg PereiramgoinClaude Opus 5.5
 		90ba2f34b5 [Build] Find compatible published wheels for precompiled installs (#58871) 
Signed-off-by: greg pereira <grpereir@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 08:05:54 -07:00
Divakar Verma c5c1161382 [ROCm][Build] Fail fast when setup.py develop deps aren't pip-preinstalled (#59447) 
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
 2026-10-08 15:28:38 -05:00
Harry MellorandClaude Opus 5.5 21d9758459 Remove Python 3.10 support now that it is EOL (#60402) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 02:30:52 -07:00
Tarun Kumar 6ad2e06c4a fix: exclude flashinfer-jit-cache from wheel install_requires (#60052) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-10-06 06:31:13 +00:00
Bugen ZhaoandClaude Opus 5.5 b3082d3044 [Rust Frontend] Remove the PyO3 tool-parser bridge (#59744) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-06 05:46:27 +00:00
Priyansh JainCursorAndreas Karatzas
 		5963795ec7 [Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508) 
Signed-off-by: priyansh jain <priyansh.jain2@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-24 02:05:44 -07:00
Ziyang MaBugen ZhaoCodex
 		d7e755c230 [Bugfix] Fix wrong vLLM version reported by pip install (proto-v* tag collision) (#57295) 
Signed-off-by: RyanMa29 <ziyang.ma@intel.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-17 08:50:14 -07:00
Harry MellorandClaude Opus 5 651a88c09a [CI] Ignore ruff D209, rejoin the docstrings it split, and silence incompatible-rule warnings (#57212) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 10:57:24 -07:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
wangxiyuan b0898a4937 [Misc] Allow install empty package from pip (#41074) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-09-16 02:02:37 +00:00
Jiangyun ZhuKimi CodeRoger Wang
 		fa008bdccf [Perf][Kernel] Integrate DeepSelect TopK for the DSA sparse indexer (#56464) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Kimi Code <noreply@moonshot.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-09-13 00:44:37 -07:00
Tarun Kumar 3140531773 Include chat template fallbacks in package_data (#53762) 
Signed-off-by: Tarun Kumar <takumar@redhat.com>
 2026-09-02 09:44:06 +00:00
Luke AlonsoandLucas Wilkinson 0d4ad47981 [Kernel] Add B12X causal paged attention backend (#52017) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2026-09-01 10:42:02 -04:00
Jung Jiyu 3bb19cd8a3 [Bugfix][CI/Build] Fail closed when selected precompiled CUDA variant is unavailable (#52545) 
Signed-off-by: Jung Jiyu <jiyu.jung@navercorp.com>
 2026-08-28 17:00:48 +08:00
Luke Alonsoandmgoin 2740c817ff [Kernel] Add b12x FP4 MoE backend (#52018) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Misha Goin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-08-21 08:04:36 -07:00
Bugen ZhaoNick HillAndreas Karatzas
 		c89d692cb5 [Build] Propagate vLLM version to Rust binaries (#52593) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-17 23:37:37 -07:00
Aarushi JainAndreas KaratzastjtanaaAndreas Karatzas
 		59d1af5698 Detect ROCm wheel variant from environment for precompiled wheels. (#49365) 
Signed-off-by: aarushjain29 <Aarushi.Jain2@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
 2026-08-13 21:19:20 -07:00
Luke Alonso 3c79b1a8bf [Kernel] Add B12X dense linear backends (#52016) 
Signed-off-by: Luke Alonso <lalonso@gmail.com>
 2026-08-13 19:48:33 -07:00
Priyansh JainandAndreas Karatzas 2acb055ed7 [CPU][Zen] Route BF16 MoE inference through zentorch on AMD (#44201) 
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-10 23:20:05 -07:00
Michael Goin 3a749ce816 [Build] Skip precompiled wheel fetch during metadata hooks (#51424) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-08-10 09:38:53 -07:00
rongfu.lengNick Hillmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		a07086e403 [Misc] Upgrade fastsafetensors version, fix metadata is null (#50827) 
Signed-off-by: lengrongfu <lenronfu@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-08-06 14:48:56 -07:00
Rehan KhanandLi, Jiang <jiang1.li@intel.com> 0187f4c88e [CPU] Enable tcmalloc for s390x (#50841) 
Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
 2026-08-05 10:55:39 +08:00
Shangdi Yu 963a658674 Bump Helion to 1.4.0 (#50307) 
Signed-off-by: Shangdi Yu <shangdiy@meta.com>
 2026-07-31 11:06:49 -07:00
Yan MaKunshang JiCyrus Leung
 		5b29c958c7 [XPU] upgrade to torch 2.13 (#48677) 
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2026-07-29 01:24:56 -07:00
+12
Jiangyun ZhuIsotr0pyThien TranBugen ZhaoYongye ZhuZiming HuangRoger WangIsotr0pyaoshen02Woosuk KwonJee Jee Liaoshen02Summer YangKevin H. LuuBowen WanggnovackNick HillYifan QiaoxiaozhoupyRoy WangJeff MaCodex
 		7c6729b769 [Model] Add Kimi K3 support: model files and kernels [1/N] (#50089) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Ziming Huang <zelda.huanghuang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: aoshen02 <aoshen@inferact.ai>
Co-authored-by: Summer Yang <girasoleyang@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
Co-authored-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: gnovack <novackgm@gmail.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: xiaozhoupy <peiyuanzhou1994@gmail.com>
Co-authored-by: Roy Wang <yasong.wang@inferact.ai>
Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
 2026-07-29 14:10:58 +08:00
Michael GoinandOpenAI Codex cc25f028b7 [Loader] Improve InstantTensor loading (#46868) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-07-17 16:30:02 -04:00
Wentao Ye 626c90b2d5 [Refactor] Move fla to third party (#48500) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-07-16 19:22:36 +01:00
Woosuk KwonBugen ZhaoGiancarlo DelfinIsotr0pyIsotr0pyJee Jee LiRoger WangYifan QiaoClaude Fable 5OpenAI Codex
 		6570c9800c [Model] Add Inkling model support [1/N] (#48799) 
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Giancarlo Delfin <32987265+TheEpicDolphin@users.noreply.github.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Isotr0py <mozf@inferact.ai>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-15 23:40:07 -07:00
Ameen PatelAmeenPJee Jee LiClaudePavani Majety
 		ba47bb5be1 Bump flashinfer version to 0.6.14 (#47669) 
Signed-off-by: AmeenP <ameenp360@gmail.com>
Signed-off-by: AmeenP <ameen@primeintellect.ai>
Co-authored-by: AmeenP <ameen@primeintellect.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Pavani Majety <pmajety@nvidia.com>
 2026-07-15 21:00:40 -07:00
ViranjanPagarClaude Opus 4.8Isotr0pyIsotr0pyRoger Wang
 		e23b19309b Deepstream video backend (#42424) 
Signed-off-by: Viranjan Pagar <vpagar@nvidia.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-07-10 02:23:30 -07:00
Varun Sundar RabindranathandVarun Sundar Rabindranath c8fb2963bd [FS-Offloading] Batch Lookup in C (#46713) 
Signed-off-by: <>
Co-authored-by: Varun Sundar Rabindranath <varun-sundar-rabindranath@h100-01.nemg-001.lab.rdu2.dc.redhat.com>
 2026-06-29 09:28:32 -07:00
Yongye ZhuClaude Opus 4.8Thien Tran
 		6691f087a6 [Minimax-M3] BF16/FP8 Indexer using MSA (#45892) 
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
 2026-06-23 10:28:49 -07:00
Chris Leonard b9a7cd464c [12/n] final _C library kernel migration (#45415) 2026-06-19 06:57:26 -07:00
Thien TranandYongye Zhu 4c62663315 [M3] Enable FP8 sparse GQA (#45744) 
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
 2026-06-16 21:38:03 -07:00
Andreas Karatzas 7e179e4bc0 [ROCm][CI] Gate incompatible HF references on Transformers v5 (#41532) 
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-16 20:34:11 +08:00
youkaichaoOpenAI CodexIsotr0pyThien TranBugen ZhaoJee Jee LiRoger WangfunctionstackxYongye ZhuJee Jee Li
 		0a1c5034f5 [Model] Add MiniMax M3 support (#45381) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Jee Jee Li <jeejeelee@inferact.ai>
 2026-06-16 01:01:25 +08:00
Isotr0py 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 12:02:21 -07:00
Xiaohong (Sean) ChenYanan Caomergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		2ec6594db9 [Kernel][Helion][1/N] Add Helion kernel for per_token_group_fp8_quant (#36902) 
Signed-off-by: Sean Chen <seachen@redhat.com>
Co-authored-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-11 08:59:08 -07:00
jasen ef67071b21 [Build] Skip spinloop extension on Python < 3.11 (#44783) 
Signed-off-by: Jasen2201 <yajizhan@amd.com>
 2026-06-11 11:23:21 +00:00
Chris LeonardandShengqi Chen 6e64c1bab1 [10c/n] Migrate MoE kernels to torch stable ABI (#44565) 
Signed-off-by: Chris Leonard <chleonar@redhat.com>
Co-authored-by: Shengqi Chen <harry-chen@outlook.com>
 2026-06-10 23:02:26 -07:00
Bugen Zhao 43914dd743 [Rust Frontend] Add Python bridge for Rust tool parsers (#44624) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-11 04:51:06 +00:00
Bugen Zhao fdfb2566c0 [Rust Frontend] [CI] Unify Rust artifact builds with setuptools-rust (#44981) 
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
 2026-06-10 17:48:34 +08:00
Turner Jabbour 0c96dd64fb [ROCm] Bump fastsafetensors to v0.3.2 from PyPI, remove git source build (#43625) 
Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com>
 2026-06-04 07:30:57 -07:00
Aakar DwivediRHarshal AdhavCursorMichael Goin
 		3fd9d2d357 [CPU][Zen] Route W8A8 and W4A16 linear inference through zentorch on AMD Zen CPUs (#41813) 
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Harshal Adhav <harshal.adhav@amd.com>
Signed-off-by: Aakar Dwivedi <aadwived@amd.com>
Co-authored-by: R <Ganesh.R@amd.com>
Co-authored-by: Harshal Adhav <harshal.adhav@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2026-05-30 14:17:21 -05:00