[Doc] Document ECMooncakeConnector for EPD disaggregation (#59196)

Signed-off-by: Teng Ma <stmatengss@gmail.com>
Co-authored-by: Teng Ma <stmatengss@users.noreply.github.com>
This commit is contained in:
Teng Ma
2026-10-02 12:10:33 +08:00
committed by GitHub
co-authored by Teng Ma
parent 8486f1ebfe
commit 1a9eaa3a68
5 changed files with 174 additions and 5 deletions
@@ -57,7 +57,12 @@ The vllm instances and `disagg_encoder_proxy` supports local URIs with ```{"url"
## EC connector and KV transfer
The `ECExampleonnector` is used to store the encoder cache on local disk and facilitate transfer. To enable the encoder disaggregation feature, add the following configuration:
`ECExampleConnector` stores encoder cache on local disk. For RDMA/TCP transfer
without a shared filesystem, use `ECMooncakeConnector` (1E + 1PD only through
this proxy). See
[ECMooncakeConnector Usage Guide](../../../docs/features/mooncake_ec_connector_usage.md).
To enable filesystem encoder disaggregation, add:
```bash
# Add to encoder instance:
@@ -104,6 +109,8 @@ If you enable prefill instance (`--prefill-servers-urls` not disabled), you will
| `--encode-servers-urls` | Comma-separated list of encoder endpoints. Every multimodal item extracted from the request is fanned out to one of these URLs in a round-robin fashion. |
| `--prefill-servers-urls` | Comma-separated list of prefill endpoints. Set to `disable`, `none`, or `""` to skip the dedicated prefill phase and run E+PD (encoder + combined prefill/decode). |
| `--decode-servers-urls` | Comma-separated list of decode endpoints. Non-stream and stream paths both round-robin over this list. |
| `--ec-consumer-zmq-addrs` | Mooncake EC consumer control addresses, aligned with `--decode-servers-urls`. Required for `ECMooncakeConnector`. Supported only in E+PD mode (`--prefill-servers-urls disable`). |
| `--ec-consumer-dp-size` | Data-parallel replicas per EC consumer (default 1). |
| `--host`, `--port` | Bind address for the proxy itself (defaults: `0.0.0.0:8000`). |
### Dynamic registration