[WideEP] Update NCCL to 2.30.7 to enable DeepEPv2 in the vllm/vllm-openai image (#45321)
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> Signed-off-by: Tyler Michael Smith <tyler@vllm.ai> Signed-off-by: Tyler Michael Smith <tyler@tylermsmith.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Ilya Markov <ilmarkov@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Codex <noreply@openai.com>
This commit is contained in:
co-authored by
Claude
Ilya Markov
OpenAI Codex
Codex
parent
e222c33f2f
commit
5d8e90a966
@@ -11,6 +11,38 @@ Step 2 is necessary for multi-node deployment.
|
||||
|
||||
All scripts accept a positional argument as workspace path for staging the build, defaulting to `$(pwd)/ep_kernels_workspace`.
|
||||
|
||||
## NCCL version requirement (CUDA 13+)
|
||||
|
||||
DeepEPv2 uses the NCCL GIN (GPU-Initiated Networking) backend, which requires
|
||||
NCCL >= 2.30.4 at both compile time and runtime. PyTorch 2.11 pins
|
||||
`nvidia-nccl-cu13==2.28.9` as a transitive dependency, so you need to
|
||||
override it.
|
||||
|
||||
**With uv** (recommended):
|
||||
|
||||
```bash
|
||||
# Create an override file
|
||||
echo "nvidia-nccl-cu13>=2.30.4" > /tmp/nccl-override.txt
|
||||
export UV_OVERRIDE=/tmp/nccl-override.txt
|
||||
|
||||
# All subsequent uv pip install commands will respect the override
|
||||
uv pip install vllm
|
||||
```
|
||||
|
||||
**With pip**:
|
||||
|
||||
```bash
|
||||
pip install vllm
|
||||
pip install "nvidia-nccl-cu13>=2.30.4" --no-deps
|
||||
```
|
||||
|
||||
The override / reinstall must happen before building DeepEP (for GIN device
|
||||
headers) and must remain in place at runtime. You can verify with:
|
||||
|
||||
```bash
|
||||
python -c "from vllm.utils.import_utils import has_deep_ep_v2; print(has_deep_ep_v2())"
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user