[Kernel] Enable Kimi-K3 SiTU on the CuteDSL MoE backend and the SM107 low-latency GEMM plan (#54606)
Signed-off-by: Bolin Sun <bolins@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
98ed0856f3
commit
d410fc12f3
@@ -1,6 +1,6 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
"""Kimi-K3 decode GEMM selection for unquantized BF16 on SM90/SM100/SM103.
|
||||
"""Kimi-K3 decode GEMM selection for unquantized BF16 on SM90/SM100/SM103/SM107.
|
||||
|
||||
Dispatch is purely by local ``(N, K)`` shape and token count ``M`` — the module
|
||||
name plays no role. Each measured shape maps to a :class:`ProjectionSpec`
|
||||
@@ -12,7 +12,10 @@ The supported capabilities carry separate measured tables:
|
||||
:data:`KIMI_K3_PROJECTIONS` was tuned on B300 (SM103),
|
||||
:data:`KIMI_K3_PROJECTIONS_SM100` on B200 (SM100), and
|
||||
:data:`KIMI_K3_PROJECTIONS_SM90` on H200 (SM90). The per-(shape, M) winners
|
||||
genuinely differ between the parts, so the tables must not be merged.
|
||||
genuinely differ between the parts, so the tables must not be merged. SM107
|
||||
(Rubin) reuses the SM103 table: the plan was validated end-to-end on SM107
|
||||
hardware, but the per-M crossovers have not been re-measured there and may
|
||||
deserve their own table once retuned.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -646,9 +649,14 @@ def _is_sm103() -> bool:
|
||||
return current_platform.is_device_capability((10, 3))
|
||||
|
||||
|
||||
def _is_sm107() -> bool:
|
||||
return current_platform.is_device_capability((10, 7))
|
||||
|
||||
|
||||
def _low_latency_table() -> dict[tuple[int, int], ProjectionSpec] | None:
|
||||
"""Measured dispatch table for the current device, or None if unsupported."""
|
||||
if _is_sm103():
|
||||
if _is_sm103() or _is_sm107():
|
||||
# SM107 reuses the SM103 table; see the module docstring.
|
||||
return KIMI_K3_PROJECTIONS
|
||||
if current_platform.is_device_capability((10, 0)):
|
||||
return KIMI_K3_PROJECTIONS_SM100
|
||||
|
||||
Reference in New Issue
Block a user