From 0e14198a63c03f899a10f3e782e88eca7f11265b Mon Sep 17 00:00:00 2001 From: Woosuk Kwon Date: Wed, 2 Sep 2026 17:58:55 -0700 Subject: [PATCH] [Skills] Add kernel benchmark sanity references (#54995) Signed-off-by: Woosuk Kwon --- .agents/skills/kernel-microbenchmark/SKILL.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/.agents/skills/kernel-microbenchmark/SKILL.md b/.agents/skills/kernel-microbenchmark/SKILL.md index 545a79f9fa..f2dd5be929 100644 --- a/.agents/skills/kernel-microbenchmark/SKILL.md +++ b/.agents/skills/kernel-microbenchmark/SKILL.md @@ -42,6 +42,25 @@ description: Build, debug, and interpret vLLM GPU kernel microbenchmarks for CUD - Keep metadata setup, plan construction, allocation, random input generation, and logging outside the timed region unless that overhead is the experiment. +## Sanity-Check Reference Numbers + +Use these as rough reference points for large, well-shaped workloads, not as +gold standards, guaranteed peaks, or hard limits. Hardware SKU, clocks, shape, +precision conventions, and the FLOP/byte accounting can move the result. A +large gap is a prompt to investigate, not proof that a kernel is poor. + +| Kernel regime | Hardware | Rough reference | +| --- | --- | ---: | +| Memory-bound, large batch | Blackwell | 6 TB/s | +| BF16 GEMM | Blackwell | 2 PFLOP/s | +| BF16 attention | B200 | 1.6 PFLOP/s | +| FP8 GEMM | Blackwell | 4 PFLOP/s | +| FP8 attention | B300 | 2.8 PFLOP/s | + +The BF16 attention reference is approximately the 1613 TFLOP/s result reported +by the FlashAttention-4 paper. Compare kernels only with matching workload and +throughput conventions. + ## Multi-GPU Benchmarks - State whether the run is local or multi-node and report the GPU topology,