[WideEP] Explain how to fix a too-old NCCL for DeepEP v2 (#60449)

Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
Misha Goin
2026-10-09 12:34:58 -07:00
committed by GitHub
co-authored by Claude
parent 04462732a4
commit 3766a67a32
4 changed files with 70 additions and 31 deletions
+18 -10
View File
@@ -11,12 +11,16 @@ Step 2 is necessary for multi-node deployment.
All scripts accept a positional argument as workspace path for staging the build, defaulting to `$(pwd)/ep_kernels_workspace`.
## NCCL version requirement (CUDA 13+)
## NCCL version requirement for DeepEPv2
DeepEPv2 uses the NCCL GIN (GPU-Initiated Networking) backend, which requires
NCCL >= 2.30.4 at both compile time and runtime. PyTorch 2.11 pins
`nvidia-nccl-cu13==2.28.9` as a transitive dependency, so you need to
override it.
DeepEPv2 (`--all2all-backend deepep_v2`) uses the NCCL GIN (GPU-Initiated
Networking) backend, which requires NCCL >= 2.30.4 both when DeepEP is built
and at runtime. PyTorch pins an older release as a dependency (PyTorch 2.13
pins `nvidia-nccl-cu13==2.29.7`), so a plain `pip install` or `uv pip install`
of vLLM is not enough. The vLLM Docker images already override it.
Upgrade NCCL before running `install_python_libraries.sh`. Use
`nvidia-nccl-cu12` on CUDA 12.
**With uv** (recommended):
@@ -29,18 +33,22 @@ export UV_OVERRIDE=/tmp/nccl-override.txt
uv pip install vllm
```
**With pip**:
Keep `UV_OVERRIDE` set for later installs into the same environment. Without
it, any `uv pip install` that resolves PyTorch again restores the pinned NCCL.
**With pip**, or to fix an existing environment:
```bash
pip install vllm
pip install "nvidia-nccl-cu13>=2.30.4" --no-deps
```
The override / reinstall must happen before building DeepEP (for GIN device
headers) and must remain in place at runtime. You can verify with:
`install_python_libraries.sh` warns when the installed NCCL is too old. If
DeepEP was built against an older NCCL, upgrade NCCL and run the script again.
When NCCL is too old at runtime, vLLM logs the reason at startup and
`deepep_v2` fails with the same message. You can check with:
```bash
python -c "from vllm.utils.import_utils import has_deep_ep_v2; print(has_deep_ep_v2())"
python -c "from vllm.utils.import_utils import deep_ep_v2_unavailable_reason; print(deep_ep_v2_unavailable_reason() or 'deepep_v2 is available')"
```
## Usage