[KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625)
Signed-off-by: aoshen02 <aoshen@inferact.ai> Signed-off-by: Dakai An <dakaian108@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Dakai An <dakaian108@gmail.com> Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
Dakai An
Dakai An
parent
f4b8d43a68
commit
3ccc0a90b6
@@ -398,7 +398,7 @@ The `kv_load_failure_policy` setting controls how the system handles failures wh
|
||||
|
||||
### For NVIDIA GB-series GPUs
|
||||
|
||||
GB-series GPUs support multi-node NVLink. NIXL supports this capability, but KVCache must be registered as VMM during KVCache registration. To enable this feature, you need to set `--enable-cumem-allocator` or `--enable-sleep-mode` flags, and set `UCX_CUDA_IPC_ENABLE_MNNVL: 'y'` env. Otherwise, NIXL can only use RDMA/TCP for cross-node KVCache transfers.
|
||||
GB-series GPUs support multi-node NVLink. NIXL supports this capability, but KVCache must be registered as VMM during KVCache registration. To enable this feature, you need to set the `--enable-cumem-allocator` flag, and set `UCX_CUDA_IPC_ENABLE_MNNVL: 'y'` env. Otherwise, NIXL can only use RDMA/TCP for cross-node KVCache transfers.
|
||||
|
||||
## Experimental Feature
|
||||
|
||||
|
||||
Reference in New Issue
Block a user