iHore/vLLM
8
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
23,331 Commits 3 Branches 0 Tags
6a95f62e2fbfebca86ec1e476e3b2e6fd8a574ab
Commit Graph
85 Commits
Author SHA1 Message Date
MOVZXandHinata e61dcb9143 [Mini-vLLM] SM86-only cut: models, backends, Rust, tests 
- Keep only Qwen3.5, Qwen3.5-MTP and Qwen4Exp model paths; trim the model registry to 41 entries
- Delete idle attention backends (MLA, FlashInfer, ROCm-only, quant backends) and dead GDN CuTeDSL kernels
- Remove inc/quark quant loaders; keep compressed-tensors and Marlin
- Drop the Rust frontend, pooling entrypoints, and idle test suites
- Rebuild kernels for sm_80;sm_86 and strip the extensions
- Verified: full import sweep clean (optional deps only) and a dummy-load Qwen3.5 engine run on RTX 3090 via the Triton GDN path

Co-authored-by: Hinata <hinata@idihore.id>
Signed-off-by: MOVZX <movzx@yahoo.com>
 2026-10-18 19:19:56 +07:00
yhcheong (vllmellm) 16d4ac4ba3 [ROCm][MoE] Support MiMo-V2.6 MXFP4 on gfx942 (#58262) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-09-30 17:23:00 +08:00
Aarushi JainandAndreas Karatzas e0f03eb4f4 [CI] Share BF16 baselines across quantization comparison tests (#58469) 
Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-09-23 19:26:09 -05:00
Nils MattesonTaneem IbrahimelehayymAndreas Karatzasmergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 		13058faa49 [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) 
Signed-off-by: Nils Matteson <nilsmatteson@icloud.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-09-21 23:18:49 +00:00
vllmellm 211e252d0b [Bugfix][Model] Fix MiMo-V2.5 fused fp8 qkv_proj sharding (pre-shard count is num_key_value_heads; MTP path too) (#57508) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2026-09-19 06:05:00 -07:00
Rohan PotdarandClaude 213c379fca [ROCm][Bugfix] Gate AITER MXFP8 MoE on the aiter enable flag (#57426) 
Signed-off-by: Rohan Potdar <rohan.potdar@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-18 00:06:05 +00:00
Harry MellorandClaude Opus 5 8c1557a79c Add pydocstyle to the ruff rules (#52136) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Harry Mellor <hej.mellor@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-16 16:44:29 +00:00
Michael LazosandOpenAI Codex 28a73ccfba [Kernel] Add NVFP4 support to the torch linear backend (#53319) 
Signed-off-by: Michael Lazos <mlazos@meta.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-09-08 20:30:05 -04:00
Roderick Wuandmgoin 2c9d68f80b [Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering (#54809) 
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com>
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2026-09-08 12:45:12 -07:00
afriedriandAndreas Karatzas dc9114b201 [ROCm][MoE] Split AITER CK and Triton MXFP4 W4A16 into separate backends (#50622) 
Signed-off-by: afriedri <afriedri@amd.com>
Signed-off-by: Andy Friedrich <afriedri@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-31 23:14:14 -05:00
Isotr0py 53e211d292 [CI/Build] Reduce more duplicate runner startup in tests (#52570) 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-17 05:19:07 +00:00
Isotr0py 073c510c91 [Migration] Migrate bitsandbytes support to OOT plugin (#43529) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
 2026-08-09 10:27:31 +08:00
fxmarty-amdandAndreas Karatzas 385d4c084e [ROCm][AITER] Hotfix for memory access fault errors in AITER triton MOE routing (#50859) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-08-03 21:07:46 -05:00
wangxiyuan dd11df04f3 [Misc] Remove deprecated calculate_kv_scales runtime KV scale calculation (#49389) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2026-08-03 11:15:13 -04:00
+1
Juan VillamizarDouglas LehrClaude Opus 4.7DanielJaden MathiasjpvillamrootDouglas LehrdllehrDaniel ChanAndreas Karatzas
 		4f1da84eb5 Enable gfx1250 ROCm architecture (#46516) 
Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com>
Signed-off-by: jpvillam <juan.villamizar@amd.com>
Signed-off-by: Daniel <danichan@amd.com>
Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>
Signed-off-by: jpvillam <jpvillam@amd.com>
Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>
Signed-off-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Signed-off-by: Jaden Mathias <Jaden.Mathias@amd.com>
Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel <danichan@amd.com>
Co-authored-by: Jaden Mathias <jaden.mathias@amd.com>
Co-authored-by: jpvillam <jpvillam@amd.com>
Co-authored-by: root <root@ctheliosr-1b114-f01-2.mnb.dcgpu>
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com>
Co-authored-by: dllehr <douglehr@amd.com>
Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-31 01:41:12 +00:00
Martin HickeyandHarry Mellor 38a267cdd5 [MyPy][1/N] Fix mypy errors in some tests/ directories and enforce follow-imports=silent (#49570) 
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-07-30 05:02:51 -07:00
fxmarty-amd 5369f7b7b8 [MXFP8][ROCm] Fix MXFP8 MoE backend selection (#49747) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-07-29 07:57:20 +08:00
Harry MellorandAndreas Karatzas ca0defa343 Make bare hugging_face imports forbidden (#49726) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-07-25 04:18:39 -07:00
Matthew Bonanniandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> bd091079cb [Attention] FlashAttention 4 SM100 FP8 kv cache support (#42569) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-07-20 10:53:27 -07:00
Kevin H. LuuSimon MoOpenAI CodexOpenAI Codex
 		1dcbbd9cac [CI] Move compatible 1xL4 jobs to H200 35GB MIG (#43024) 
Signed-off-by: Simon Mo <simon@inferact.ai>
Co-authored-by: Simon Mo <simon@inferact.ai>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
 2026-07-19 19:21:25 -07:00
fxmarty-amdandAndreas Karatzas 03c6d01c30 [OCP MX ] Add back emulation to available OCP MX backends list (#46629) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
 2026-06-28 12:43:19 -05:00
JartXandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 70749fdcca [Feature] Triton INT4 per-token-head KV cache quantization (#40835) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-24 10:21:25 +00:00
fxmarty-amd e368415daa [AMD][OCP MX][CI] Fix tests to not dispatch on UNFUSED_TRITON backend on MI300, improve w_mxfp4_a_fp8 emulation support (#46142) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
 2026-06-23 14:25:27 -04:00
Mohammad Miadh Angkad d1a38c2762 [Kernel][Performance] Add FlashInfer cutedsl NVFP4 GEMM backend (#42235) 
Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-06-22 16:17:18 -04:00
Isotr0py 6635279d8a [Migration] Migrate GGUF quantization support to plugin (#39612) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-06-12 12:02:21 -07:00
Ting SUN c1076839c9 [Bugfix][Model] Pass revision by name in Run:ai and bitsandbytes index downloads (#45308) 
Signed-off-by: Ting Sun <suntcrick@gmail.com>
 2026-06-11 20:21:46 -07:00
Harry Mellor 03878d1c22 Deprecations for v0.23 and v0.24 (#44992) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-06-11 14:35:38 +00:00
Walter Beller-Moralesandmergify[bot] <37929162+mergify[bot]@users.noreply.github.com> 753e9d55e6 [Quantization] add online fp8 ptpc (#44132) 
Signed-off-by: walterbm <walter.beller.morales@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2026-06-08 22:42:11 +08:00
Liangliang MaandKunshang Ji e9499996df [BugFix][Platform] Fix import vllm.platforms.rocm error on non-CUDA test_gpt_oss.py (#43571) 
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-05-29 23:16:49 -07:00
Wentao Ye 33d7cbe02c [Model Runner v2] Force v1 runner for tests (#43233) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2026-05-23 16:37:24 -07:00
Michael Goin b2a27b82d9 [Kernel][UX] Add --linear-backend arg for linear kernel selection (#39538) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-05-15 17:07:39 -07:00
Lanze LiuandMatthew Bonanni b2c58ee942 [FlashAttn] Fix supports_kv_cache_dtype() accepting unhandled fp8 kv-cache dtype variants (#42685) 
Signed-off-by: Lanze Liu <lanzetech@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-05-15 15:34:59 -04:00
Chengyi NieChengyi NieCursor
 		fa2a33b893 [Quant] Consolidate GPTQ: rename gptq_marlin.py to auto_gptq.py (#38288) 
Signed-off-by: Chengyi Nie <cnie@roblox.com>
Co-authored-by: Chengyi Nie <cnie@roblox.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-15 08:25:52 +08:00
SoluMilken 25abddc1a5 [BugFix] Fix Gemma4 'layers.0.moe.experts.0.down_proj_packed' KeyError issue (#40708) 
Signed-off-by: SoluMilken <ypiheyn.imm02g@g2.nctu.edu.tw>
 2026-05-09 17:20:44 +00:00
fxmarty-amdandKyle Sayers d622e27d2b [NVFP4] NVFP4 MOE emulation fallback for H100/MI300/MI350, standardize TritonExperts usage for OCP MX emulation (#35737) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-04-22 08:58:54 -07:00
fxmarty-amdandKyle Sayers 00d7b497b3 [NVFP4] Support NVFP4 dense models from modelopt and compressed-tensors on AMD Instinct MI300, MI355X and Hopper through emulation (#35733) 
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
Signed-off-by: fxmarty-amd <felmarty@amd.com>
Co-authored-by: Kyle Sayers <kylesayrs@gmail.com>
 2026-04-06 16:18:27 -06:00
JartXyangyang4991Wentao YeIsotr0py
 		2ce3d0ce36 [Feature] KV cache per-token-head INT8/FP8 quantization (#38378) 
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: yangyang4991 <yangyang4991@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
 2026-04-02 08:13:26 -04:00
Michael Goin db5d0719e1 [Kernel] Add MXFP8 to Marlin GEMM/MoE and refactor Mxfp8LinearOp (#34664) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2026-04-01 09:41:42 -07:00
wliao2Harry MellorKunshang Ji
 		4dfad17ed1 replace cuda_device_count_stateless() to current_platform.device_count() (#37841) 
Signed-off-by: Liao, Wei <wei.liao@intel.com>
Signed-off-by: wliao2 <wei.liao@intel.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
 2026-03-31 22:32:54 +08:00
EdalatiAli e5b807607c [Quant][Feature] Support online MXFP8 quantization for MoE and dense models (#35448) 
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
 2026-03-16 18:07:39 -04:00
Marc Sun c973ecdead [bnb] Skip moe + bnb test (#36896) 
Signed-off-by: Marc Sun <marc@huggingface.co>
 2026-03-12 18:03:25 +00:00
Micah Williamson fc4657756f [ROCm][CI] Enable AITER for failing test_gpt_oss test case on MI355 (#36174) 2026-03-07 13:50:17 -08:00
Micah Williamson e7213003cb [ROCm][CI] Fix TP size issue for test_gpt_oss (#35887) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-03-03 20:57:34 +00:00
Micah Williamson 8b9e8b7454 [ROCm][CI] Fix Assertion Logic For test_gpt_oss (#35806) 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
 2026-03-03 05:08:04 +00:00
xuebwang-amdRobert ShawRobert Shaw
 		b129136c7a [ROCm][Quantization] GPT_OSS in amd-quark format model loading and emulations (#29008) 
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2026-02-10 10:08:05 -05:00
Cyrus Leung edb359cce4 [Renderer] Define render_cmpl and render_chat (#34039) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2026-02-07 05:24:40 -08:00
Robert ShawRobert ShawHarry Mellor
 		af9b69f977 [Quantization][Deprecation] Remove Marlin 24 (#32688) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2026-01-28 15:54:59 +00:00
Robert ShawandRobert Shaw 247d1a32ea [Quantization][Deprecation] Remove BitBlas (#32683) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
 2026-01-28 11:06:22 +00:00
Roberto L. Castro 8ef50d9a6b [Kernel][Performance] Enable smaller Scaling Factor tiling for NVFP4 small-batch decoding (#30885) 
Signed-off-by: LopezCastroRoberto <roberto.lopez.castro@udc.es>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
 2026-01-13 15:22:53 -08:00
Matthew Bonanni 2612ba9285 [1/N][Attention] Restructure attention: move files (#31916) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2026-01-09 13:10:24 -08:00