[2/N][Feat][Perf] Add new warmup infrastructure for JITs. Add predicate filtering for JIT warmup, and migrate Inkling FA4 (#49315)

Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>
Signed-off-by: Roberto L. Castro <38211239+LopezCastroRoberto@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
Roberto L. Castro
2026-08-11 07:46:07 -04:00
committed by GitHub
co-authored by Lucas Wilkinson mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent d8f840071e
commit 6c95a641e9
12 changed files with 739 additions and 386 deletions
@@ -2,8 +2,8 @@
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
"""Correctness test for the Inkling FA4 relative-attention score-mod kernel.
Checks ``inkling_fa4_rel_attention`` against a pure-PyTorch reference that
implements the relative bias exactly as documented in the Inkling architecture
Checks ``INKLING_FA4_REL_ATTENTION_KERNEL`` against a pure-PyTorch reference
that implements the relative bias exactly as documented in the Inkling architecture
guide::
logit(i, j, h) = (1 / head_dim) * dot(q[i, h], k[j, h]) + rel_bias(i, j, h)
@@ -23,9 +23,9 @@ from vllm.models.inkling.nvidia.attention import (
compute_log_scaling_tau,
)
from vllm.models.inkling.nvidia.ops.fa4_rel_attention import (
INKLING_FA4_REL_ATTENTION_KERNEL,
_use_sheared_bias,
inkling_fa4_num_splits,
inkling_fa4_rel_attention,
)
from vllm.platforms import current_platform
from vllm.platforms.interface import DeviceCapability
@@ -279,7 +279,7 @@ def _run_case(seq_lens, num_heads, num_kv_heads, rel_extent, window_left, seed=0
num_kv_heads=num_kv_heads,
max_kv_len=max(kv_lens),
)
out = inkling_fa4_rel_attention(
out = INKLING_FA4_REL_ATTENTION_KERNEL(
q,
key_cache,
value_cache,