MOVZX and Hinata
5f3c7bd8ab
[Mini-VLLM] Remove the ROCm device path, debug routes and arch-gated tests
...
- Delete csrc/rocm, hipify, the _rocm_C extension and the setup.py rocm
target path; drop the rocm requirement files
- Delete the aiter, rdna3, skinny-gemm and ROCm fused-MoE kernel modules
and retarget the linear kernel registries away from them
- Reduce platforms/rocm.py to the GCN architecture predicates that live
code still imports, and delete the RocmPlatform class and plugin
- Drop the VLLM_SERVER_DEV_MODE debug surface: reset, collective_rpc,
server_info and sleep routes plus the log_error_stack flag
- Delete lm-format-enforcer, lark and otel-semantic-conventions-ai pins;
the live server process maps none of them
- Delete tests that only build architectures this fork no longer
registers, and test directories left without any test file
- Restore the tests/v1/kv_connector mock helpers that scheduler tests
import at module level
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 19:31:07 +07:00
MOVZX and Hinata
81074e94a3
[Mini-vLLM] Cut dead subsystems: models, entrypoints, backends, utils, tests
...
- Drop dead model modules, keeping files live imports require
- Remove anthropic, cohere, codegen entrypoints and batch STT modes
- Delete unused attention backends, MoE experts, and NPU/XPU util leaves
- Strip dead third_party trees and ROCm leaves with no CUDA path
- Remove mooncake EC connectors: dependency missing since branch base
- Re-audit verified by full import sweep and Qwen3.5 smoke test
Co-authored-by: Hinata <hinata@idihore.id >
Signed-off-by: MOVZX <movzx@yahoo.com >
2026-10-19 00:20:48 +07:00
fe98d90a9d
[Dependency] Update Humming and remove NVFP4 scale workaround ( #61556 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Co-authored-by: Codex <codex@openai.com >
Co-authored-by: Misha Goin <mgoin64@gmail.com >
Co-authored-by: Kevin H. Luu <khluu000@gmail.com >
2026-10-14 13:32:36 -07:00
e83222396f
[ROCm][Perf] W4A16: speed up skinny GEMM decode, including speculative batches ( #57199 )
...
Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com >
2026-10-13 17:21:30 +00:00
Guancheng Fu and Kunshang Ji
acec1b1d84
[Bugfix][XPU] Remove stale W4A8 weight after repacking ( #57511 )
...
Signed-off-by: Guancheng Fu <guancheng.fu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-10-10 04:27:35 +00:00
00CC
5281e49908
[ROCm][Perf] Enable medium-skinny dispatch in RDNAHybridW4A16LinearKernel ( #52619 )
...
Signed-off-by: tangzzycc <3081129260@qq.com >
2026-10-07 13:21:43 +00:00
d1f3d8b870
[Quantization] Prefer Humming before Marlin backends on SM90 ( #56997 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 12:31:35 -07:00
Matthias Gehre
b267fe4881
[ROCm][Perf] W4A16: pad gfx11 weight and activation strides ( #56301 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-10-06 08:22:33 -05:00
e12291d733
[CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers ( #54024 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Signed-off-by: Ganesh R <Ganesh.R@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-10-01 15:18:25 +08:00
Rukhaiya2004 and Li, Jiang <jiang1.li@intel.com>
72e7874fa6
[Hardware][Power] Enable W4A16 (AWQ & GPTQ) quantization on POWER10 using VSX ( #59149 )
...
Signed-off-by: Rukhaiya <bibirukhaiya123@gmail.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-09-30 19:44:48 +08:00
HDCharles
9f07d023d0
[Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors ( #46528 )
2026-09-23 10:23:11 -04:00
Misha Goin
25965ee583
[Kernel] Remove AllSpark INT8 W8A16 GEMM backend ( #58001 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-09-22 14:07:34 -07:00
7d99c2c4fd
[Feature][Humming] Humming feature integration ( #56685 )
...
Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Codex <noreply@openai.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-20 08:11:01 -07:00
Misha Goin and ErinYin
e5fce7b56b
[Core][Kernel] Share persistent workspaces for Marlin and Humming ( #57421 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: ErinYin <liang.yin@daocloud.io >
2026-09-20 17:28:33 +08:00
Harry Mellor and Claude Opus 5
8c1557a79c
Add pydocstyle to the ruff rules ( #52136 )
...
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
Signed-off-by: Harry Mellor <hej.mellor@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-16 16:44:29 +00:00
Matthias Gehre
dc2e8f1157
[ROCm][Perf] W4A16: keep skinny GEMM zero-points packed 4-bit ( #54965 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-09-14 17:40:25 +08:00
Roderick Wu and mgoin
2c9d68f80b
[Quant][Kernel] Remove GPTQ Group/Dynamic Activation Ordering ( #54809 )
...
Signed-off-by: Roderick-Wu <roderickwu2003@gmail.com >
Signed-off-by: Roderick Wu <roderickwu2003@gmail.com >
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-09-08 12:45:12 -07:00
ee3c00bbf4
[Performance] Register Triton W4A16 GEMM as a custom op ( #51453 )
...
Signed-off-by: giuseppegrossi <ggrossi@amd.com >
Co-authored-by: TJian <tunjian.tan@embeddedllm.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-09-01 22:55:34 -05:00
qli88
1d3a8b9e22
[ROCm][Bugfix] Fix Triton W4A16 bug in determining if transpose is required for GPTQ/AutoGPTQ ( #48998 )
...
Signed-off-by: Qiang Li <qiang.li2@amd.com >
2026-08-17 12:26:05 +00:00
Jinzhen Lin and OpenAI Codex
fcde8e1460
[Refactor] refactor humming linear and moe backends to use explicit layer configs ( #49610 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: OpenAI Codex <noreply@openai.com >
2026-08-07 10:03:19 -07:00
Ryan Clark and Claude Fable 5
68fb303a41
[Bugfix] Preserve Marlin runtime tensor storage across weight reload ( #48438 )
...
Signed-off-by: Ryan Clark <ryanclark2k@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-30 14:50:03 -07:00
Giuseppe Grossi and mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
1d99f0f421
[ROCm][BugFix] Triton W4A16 handling for GPTQ/AutoGPTQ qzeros layout ( #47770 )
...
Signed-off-by: giuseppegrossi <ggrossi@amd.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-07-15 11:55:47 -05:00
HDCharles
9b2be4e9a5
[Quant] Enable humming w[2-7]a[4,8] inference with compressed-tensors ( #46390 )
...
Signed-off-by: HDCharles <charlesdavidhernandez@gmail.com >
2026-07-14 20:22:31 -06:00
Matthias Gehre and Andreas Karatzas
382bbd5144
[ROCm][Kernel] Add HybridW4A16LinearKernel: Triton prefill + HIP skinny decode ( #40977 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
Co-authored-by: Andreas Karatzas <akaratza@amd.com >
2026-07-13 20:22:00 -07:00
Jinzhen Lin and mgoin
d891b9bd51
[Quantization] add humming moe backend to all dense/moe oracles ( #41652 )
...
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com >
Co-authored-by: mgoin <mgoin64@gmail.com >
2026-07-06 13:36:07 -07:00
638b1a99cc
[CPU][RISC-V] Add RVV path for W4A8 INT4 GEMM ( #45269 )
...
Signed-off-by: wcy <233313160abc@gmail.com >
Co-authored-by: lyd1992 <liuyudong@iscas.ac.cn >
Co-authored-by: OpenAI Codex <codex@openai.com >
2026-06-25 08:18:10 +00:00
wcy and Li, Jiang <jiang1.li@intel.com>
d2c671c29b
[CPU][RISC-V] Add RVV micro GEMM for WNA16 ( #44324 )
...
Signed-off-by: wcy <233313160abc@gmail.com >
Co-authored-by: Li, Jiang <jiang1.li@intel.com >
2026-06-22 12:53:54 +00:00
8d4f54966c
fix(quantization): Fix AWQ dequantize on Intel XPU and refactor AutoAWQ config ( #42727 )
...
Signed-off-by: Alex <alex.tech.lab@outlook.com >
Signed-off-by: AlexHuang <jihuihuang@tencent.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2026-06-18 20:12:28 +08:00
Michael Goin
9eaacb23ec
[Kernel] Consolidate Marlin thread-tile padding across all dense Marlin paths ( #45295 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-06-12 13:46:21 -07:00
Divakar Verma
166d14e9bf
[bugfix] skip conch kernel for g_idx reordering ( #45072 )
...
Signed-off-by: Divakar Verma <divakar.verma@amd.com >
2026-06-10 23:04:19 +08:00
Michael Goin and Claude
87c15d46e3
[Bugfix] Lazily import the humming quantization backend ( #44921 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-06-10 06:06:17 -07:00
Michael Goin
06ee2d8433
[Quant] Support compressed-tensors WNA8O8Int linears and WNInt embeddings ( #44340 )
...
Signed-off-by: mgoin <mgoin64@gmail.com >
2026-06-04 07:40:33 -07:00
zofia and Kunshang Ji
6314de8bad
[XPU] [Bug] remove xpuw4a16 output size check ( #44168 )
...
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com >
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com >
2026-06-02 22:26:20 +08:00
3fd9d2d357
[CPU][Zen] Route W8A8 and W4A16 linear inference through zentorch on AMD Zen CPUs ( #41813 )
...
Signed-off-by: R <Ganesh.R@amd.com >
Signed-off-by: Harshal Adhav <harshal.adhav@amd.com >
Signed-off-by: Aakar Dwivedi <aadwived@amd.com >
Co-authored-by: R <Ganesh.R@amd.com >
Co-authored-by: Harshal Adhav <harshal.adhav@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-05-30 14:17:21 -05:00
JartX
0cff0741ff
[Kernel][ROCm] Native W4A16 kernel for AMD RDNA3 (gfx1100) — fp16 + bf16 ( #41394 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-05-29 11:04:40 +00:00
Li, Jiang
20d69d100a
[CPU] Migrate cpu_awq into awq_marlin ( #43841 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-05-28 22:36:31 +08:00
Luciano Martins and Luciano Martins
2272062471
[Kernel] Enable TritonW4A16LinearKernel as CUDA fallback for non-Marlin-aligned W4A16 shapes ( #43731 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
2026-05-27 18:36:27 +08:00
Kunshang Ji
36dcaf25d8
[XPU] add gptq(int4) support ( #37844 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-05-19 11:17:09 +08:00
JartX
f414f90601
[Bugfix][Kernel][ROCm] Fix triton_w4a16 scales mismatch when BLOCK_K > group_size ( #39705 )
...
Signed-off-by: JartX <sagformas@epdcenter.es >
2026-04-13 14:29:45 -04:00
d468322dc1
[Kernel][Hardware][AMD] Add TritonW4A16LinearKernel for ROCm ( #37352 )
...
Signed-off-by: jatseng-ai <jatseng@amd.com >
Signed-off-by: jatseng-ai <janet.tseng@amd.com >
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com >
2026-04-10 10:25:27 +00:00
Li, Jiang
becaed6ec8
[CPU] Support CT W4A16 on CPU MP kernel ( #38219 )
...
Signed-off-by: jiang1.li <jiang1.li@intel.com >
2026-03-27 14:15:28 +08:00
Andreas Karatzas
a8e48a7b85
[CI] Fix conch kernel crash on 3D input by reshaping to 2D before GEMM ( #38178 )
...
Signed-off-by: Andreas Karatzas <akaratza@amd.com >
2026-03-26 11:46:03 -05:00
Matthias Gehre and Michael Goin
410d300893
[ROCm][Refactor] Enable AWQMarlinConfig on ROCm to use choose_mp_linear_kernel ( #36505 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
Co-authored-by: Michael Goin <mgoin64@gmail.com >
2026-03-23 15:36:08 +08:00
Matthias Gehre
5e806bcf54
[Bugfix] Fix ConchLinearKernel channelwise quantization (group_size=-1) ( #37329 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-03-20 10:32:21 -05:00
Matthias Gehre
56a62c310c
[Bugfix] Reject channelwise quantization (group_size <= 0) in ExllamaLinearKernel ( #37331 )
...
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com >
2026-03-20 10:31:57 -05:00
Tianmu Li
47b7af0d87
[Feat] Enable CompressedTensorW4A8Int for XPU ( #37207 )
...
Signed-off-by: Li, Tianmu <tianmu.li@intel.com >
2026-03-20 02:34:28 +00:00
Andrey Talman
68f783a727
[Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility ( #35673 )
...
Signed-off-by: atalman <atalman@fb.com >
2026-03-17 18:47:59 +00:00
Jiayi Yan and Harry Mellor
6a895197fa
[Bugfix][CI] fix typos ( #34934 )
...
Signed-off-by: 1195343015 <1195343015@qq.com >
Signed-off-by: Jiayi Yan <66017932+1195343015@users.noreply.github.com >
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com >
2026-03-05 17:05:46 +00:00
Kunshang Ji
66a2209645
[Hardware] Replace torch.cuda.synchronize() api with torch.accelerator.synchronize ( #36085 )
...
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com >
2026-03-05 10:36:39 +00:00
Fadi Arafeh and Tyler Michael Smith
4171ff6dd9
[CPU][Feat] Enable KleidiAI INT8_W4A8 for all input dtypes ( #34890 )
...
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com >
2026-02-26 05:00:10 +00:00