[ROCm][Perf] W4A16: keep skinny GEMM zero-points packed 4-bit (#54965)

Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
This commit is contained in:
Matthias Gehre
2026-09-14 17:40:25 +08:00
committed by GitHub
parent 238cb2b191
commit dc2e8f1157
4 changed files with 117 additions and 45 deletions
+2 -1
View File
@@ -31,7 +31,8 @@ TORCH_LIBRARY_EXPAND(TORCH_EXTENSION_NAME, rocm_ops) {
rocm_ops.impl("wvSplitK", torch::kCUDA, &wvSplitK);
// W4A16 grouped skinny GEMM: packed int4 weights, per-group scales,
// optional zero points for asymmetric quantization
// optional zero points [M/8, K/group_size] int32 for asymmetric
// quantization
rocm_ops.def(
"wvSplitK_int4_g(Tensor in_a, Tensor in_b, Tensor in_scale, "
"Tensor? in_zero_points, Tensor? in_bias, int CuCount, "