[Frontend][Build] Add vllm download-kernels for FlashInfer precompiled kernels (#58765)
Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Misha Goin <mgoin64@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
be0823357f
commit
29e4d2d42e
+5
-7
@@ -936,13 +936,11 @@ RUN --mount=type=cache,target=/opt/uv/cache \
|
||||
fi && \
|
||||
rm /tmp/requirements-cuda.txt /tmp/common.txt
|
||||
|
||||
# Install FlashInfer JIT cache (requires CUDA-version-specific index URL)
|
||||
# https://docs.flashinfer.ai/installation.html
|
||||
# From versions.json: .flashinfer.version
|
||||
ARG FLASHINFER_VERSION=0.7.0.post1
|
||||
RUN --mount=type=cache,target=/opt/uv/cache \
|
||||
uv pip install --system flashinfer-jit-cache==${FLASHINFER_VERSION} \
|
||||
--index-url https://flashinfer.ai/whl/cu$(echo $CUDA_VERSION | cut -d. -f1,2 | tr -d '.')
|
||||
# Install FlashInfer's precompiled kernels for all GPU architectures (the default).
|
||||
# `vllm download-kernels` wraps this command but vLLM is only installed in later
|
||||
# stages, and running it there would rebuild this large layer for every wheel.
|
||||
RUN --mount=type=cache,target=/opt/pip/cache \
|
||||
PIP_CACHE_DIR=/opt/pip/cache flashinfer download-kernels
|
||||
|
||||
# ============================================================
|
||||
# OPENAI API SERVER DEPENDENCIES
|
||||
|
||||
Reference in New Issue
Block a user