[CI][ROCm] Ship RIXL with vllm/vllm-openai-rocm (#41634)
Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>
This commit is contained in:
@@ -507,6 +507,10 @@ RUN --mount=type=bind,from=export_vllm,src=/,target=/install \
|
|||||||
&& pip uninstall -y vllm \
|
&& pip uninstall -y vllm \
|
||||||
&& uv pip install --system *.whl
|
&& uv pip install --system *.whl
|
||||||
|
|
||||||
|
# Install RIXL wheel
|
||||||
|
RUN --mount=type=bind,from=build_rixl,src=/app/install,target=/rixl_install \
|
||||||
|
uv pip install --system /rixl_install/*.whl
|
||||||
|
|
||||||
ARG COMMON_WORKDIR
|
ARG COMMON_WORKDIR
|
||||||
ARG BASE_IMAGE
|
ARG BASE_IMAGE
|
||||||
ARG NIC_BACKEND
|
ARG NIC_BACKEND
|
||||||
@@ -517,6 +521,10 @@ COPY --from=export_vllm /benchmarks ${COMMON_WORKDIR}/vllm/benchmarks
|
|||||||
COPY --from=export_vllm /examples ${COMMON_WORKDIR}/vllm/examples
|
COPY --from=export_vllm /examples ${COMMON_WORKDIR}/vllm/examples
|
||||||
COPY --from=export_vllm /docker ${COMMON_WORKDIR}/vllm/docker
|
COPY --from=export_vllm /docker ${COMMON_WORKDIR}/vllm/docker
|
||||||
|
|
||||||
|
# Use legacy IPC mode for HSA to avoid GPU memory pinning issues with UCX rocm_ipc
|
||||||
|
# See: https://github.com/ROCm/rocm-libraries/issues/6266
|
||||||
|
ENV HSA_ENABLE_IPC_MODE_LEGACY=1
|
||||||
|
|
||||||
ENV TOKENIZERS_PARALLELISM=false
|
ENV TOKENIZERS_PARALLELISM=false
|
||||||
|
|
||||||
# ENV that can improve safe tensor loading, and end-to-end time
|
# ENV that can improve safe tensor loading, and end-to-end time
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ Install the NIXL library: `uv pip install nixl`, as a quick start on Nvidia plat
|
|||||||
- Refer to [NIXL official repository](https://github.com/ai-dynamo/nixl) for more installation instructions
|
- Refer to [NIXL official repository](https://github.com/ai-dynamo/nixl) for more installation instructions
|
||||||
- The specified required NIXL version can be found in [requirements/kv_connectors.txt](../../requirements/kv_connectors.txt) and other relevant config files
|
- The specified required NIXL version can be found in [requirements/kv_connectors.txt](../../requirements/kv_connectors.txt) and other relevant config files
|
||||||
|
|
||||||
For ROCm platform, the [base ROCm docker file](../../docker/Dockerfile.rocm_base) includes RIXL and ucx already.
|
For ROCm platform, the [ROCm docker file](../../docker/Dockerfile.rocm) includes RIXL and ucx already.
|
||||||
|
|
||||||
- Refer to [RIXL official repository](https://github.com/rocm/rixl) for more information
|
- Refer to [RIXL official repository](https://github.com/rocm/rixl) for more information
|
||||||
- The supportive libraries for RIXL can be found in [requirements/kv_connectors_rocm.txt](../../requirements/kv_connectors_rocm.txt)
|
- The supportive libraries for RIXL can be found in [requirements/kv_connectors_rocm.txt](../../requirements/kv_connectors_rocm.txt)
|
||||||
|
|||||||
Reference in New Issue
Block a user