[ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659)
Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com>
This commit is contained in:
co-authored by
Andreas Karatzas
parent
1f0bc49ee6
commit
35f92d2abc
@@ -1,5 +1,27 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
# Portions adapted from ROCm/aiter (https://github.com/ROCm/aiter/pull/5750):
|
||||
# SPDX-License-Identifier: MIT
|
||||
# Copyright (C) 2026, Advanced Micro Devices, Inc. All rights reserved.
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
# SOFTWARE.
|
||||
"""MXFP8 GEMM on 32x32 block-scaled weights for gfx950 (``tl.dot_scaled``).
|
||||
|
||||
``y = x @ w.T`` with an MXFP8 activation (e4m3 values, one E8M0 scale per
|
||||
@@ -18,6 +40,10 @@ Two kernels, picked per shape from a table tuned on MI355X:
|
||||
Either can split K. The partials are normally summed in the same launch by
|
||||
the last program of each output tile to finish, in split order, so the result
|
||||
does not depend on scheduling. Otherwise a second launch reduces them.
|
||||
|
||||
The packed kernel and the in-launch split-K reduction on one XCD
|
||||
(``_split_tile``, ``_sum_splits``, ``_split_counters``, ``_k_partition``) are
|
||||
adapted from the group32 GEMM in ROCm/aiter#5750.
|
||||
"""
|
||||
|
||||
from typing import NamedTuple
|
||||
|
||||
Reference in New Issue
Block a user