[WideEP] Explain how to fix a too-old NCCL for DeepEP v2 (#60449)
Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
+18
-10
@@ -11,12 +11,16 @@ Step 2 is necessary for multi-node deployment.
|
||||
|
||||
All scripts accept a positional argument as workspace path for staging the build, defaulting to `$(pwd)/ep_kernels_workspace`.
|
||||
|
||||
## NCCL version requirement (CUDA 13+)
|
||||
## NCCL version requirement for DeepEPv2
|
||||
|
||||
DeepEPv2 uses the NCCL GIN (GPU-Initiated Networking) backend, which requires
|
||||
NCCL >= 2.30.4 at both compile time and runtime. PyTorch 2.11 pins
|
||||
`nvidia-nccl-cu13==2.28.9` as a transitive dependency, so you need to
|
||||
override it.
|
||||
DeepEPv2 (`--all2all-backend deepep_v2`) uses the NCCL GIN (GPU-Initiated
|
||||
Networking) backend, which requires NCCL >= 2.30.4 both when DeepEP is built
|
||||
and at runtime. PyTorch pins an older release as a dependency (PyTorch 2.13
|
||||
pins `nvidia-nccl-cu13==2.29.7`), so a plain `pip install` or `uv pip install`
|
||||
of vLLM is not enough. The vLLM Docker images already override it.
|
||||
|
||||
Upgrade NCCL before running `install_python_libraries.sh`. Use
|
||||
`nvidia-nccl-cu12` on CUDA 12.
|
||||
|
||||
**With uv** (recommended):
|
||||
|
||||
@@ -29,18 +33,22 @@ export UV_OVERRIDE=/tmp/nccl-override.txt
|
||||
uv pip install vllm
|
||||
```
|
||||
|
||||
**With pip**:
|
||||
Keep `UV_OVERRIDE` set for later installs into the same environment. Without
|
||||
it, any `uv pip install` that resolves PyTorch again restores the pinned NCCL.
|
||||
|
||||
**With pip**, or to fix an existing environment:
|
||||
|
||||
```bash
|
||||
pip install vllm
|
||||
pip install "nvidia-nccl-cu13>=2.30.4" --no-deps
|
||||
```
|
||||
|
||||
The override / reinstall must happen before building DeepEP (for GIN device
|
||||
headers) and must remain in place at runtime. You can verify with:
|
||||
`install_python_libraries.sh` warns when the installed NCCL is too old. If
|
||||
DeepEP was built against an older NCCL, upgrade NCCL and run the script again.
|
||||
When NCCL is too old at runtime, vLLM logs the reason at startup and
|
||||
`deepep_v2` fails with the same message. You can check with:
|
||||
|
||||
```bash
|
||||
python -c "from vllm.utils.import_utils import has_deep_ep_v2; print(has_deep_ep_v2())"
|
||||
python -c "from vllm.utils.import_utils import deep_ep_v2_unavailable_reason; print(deep_ep_v2_unavailable_reason() or 'deepep_v2 is available')"
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Reference in New Issue
Block a user