[Dependency] Upgrade FlashInfer version to 0.7.0 (#58069)

Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Kevin H. Luu <khluu000@gmail.com>
This commit is contained in:
Wei Zhao
2026-09-23 22:45:23 -07:00
committed by GitHub
co-authored by Codex Yongye Zhu Kevin H. Luu
parent a1b6763df6
commit 44bf011f44
15 changed files with 197 additions and 31 deletions
+13 -3
View File
@@ -79,9 +79,9 @@ def flashinfer_sampler_supported() -> bool:
Returns False (with appropriate logging) when ``VLLM_USE_FLASHINFER_SAMPLER``
is 0, when the platform isn't CUDA, when the GPU's compute capability is
unsupported, or when FlashInfer cannot JIT-compile for the current
GPU/CUDA toolchain. Raises ``RuntimeError`` if the user explicitly opted in
via the env var but FlashInfer is unavailable.
unsupported, when the GPU has 16 or fewer SMs, or when FlashInfer cannot
JIT-compile for the current GPU/CUDA toolchain. Raises ``RuntimeError`` if
the user explicitly opted in via the env var but FlashInfer is unavailable.
Assumes flashinfer is installed, as guaranteed by ``requirements/cuda.txt``;
otherwise importing the FlashInfer backend below raises ``ImportError``.
@@ -107,6 +107,16 @@ def flashinfer_sampler_supported() -> bool:
unsupported_reason = (
f"unsupported compute capability {capability.as_version_str()}"
)
elif (
num_sms := current_platform.num_compute_units(
torch.accelerator.current_device_index()
)
) <= 16:
# FlashInfer 0.7+ rejects multi-CTA top-k masking on <=16 SMs because
# its cross-CTA software barrier cannot guarantee forward progress.
unsupported_reason = (
f"top-k masking requires more than 16 SMs; device has {num_sms}"
)
else:
unsupported_reason = _flashinfer_jit_unsupported_reason(capability)