diff --git a/docs/serving/parallelism_scaling.md b/docs/serving/parallelism_scaling.md index ed93432701..b69ca17e83 100644 --- a/docs/serving/parallelism_scaling.md +++ b/docs/serving/parallelism_scaling.md @@ -68,6 +68,12 @@ vLLM uses Ray to manage the distributed execution of tasks across multiple nodes Ray also offers high-level APIs for large-scale [offline batch inference](https://docs.ray.io/en/latest/data/working-with-llms.html) and [online serving](https://docs.ray.io/en/latest/serve/llm) that can leverage vLLM as the engine. These APIs add production-grade fault tolerance, scaling, and distributed observability to vLLM workloads. +Ray is an optional dependency. Install it explicitly before using Ray-based execution, for example: + +```bash +pip install "ray[cgraph]" +``` + For details, see the [Ray documentation](https://docs.ray.io/en/latest/index.html). ### Ray cluster setup with containers diff --git a/requirements/cuda.txt b/requirements/cuda.txt index 22477dc821..79b34a1a13 100644 --- a/requirements/cuda.txt +++ b/requirements/cuda.txt @@ -4,7 +4,6 @@ numba == 0.61.2 # Required for N-gram speculative decoding # Dependencies for NVIDIA GPUs -ray[cgraph]>=2.48.0 torch==2.10.0 torchaudio==2.10.0 # These must be updated alongside torch diff --git a/requirements/rocm.txt b/requirements/rocm.txt index fcc67e463e..a46a1b574d 100644 --- a/requirements/rocm.txt +++ b/requirements/rocm.txt @@ -10,7 +10,6 @@ numba == 0.61.2 # Required for N-gram speculative decoding # Dependencies for AMD GPUs datasets -ray[cgraph]>=2.48.0 peft pytest-asyncio tensorizer==2.10.1