[KV Connector] Support sleep mode with MooncakeConnector over RDMA (#59625)

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Signed-off-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Dakai An <dakaian108@gmail.com>
Co-authored-by: Dakai An <77474977+andakai@users.noreply.github.com>
This commit is contained in:
aoshen02
2026-10-14 04:18:47 +00:00
committed by GitHub
co-authored by Claude Fable 5.1 Dakai An Dakai An
parent f4b8d43a68
commit 3ccc0a90b6
14 changed files with 205 additions and 8 deletions
+1 -1
View File
@@ -398,7 +398,7 @@ The `kv_load_failure_policy` setting controls how the system handles failures wh
### For NVIDIA GB-series GPUs
GB-series GPUs support multi-node NVLink. NIXL supports this capability, but KVCache must be registered as VMM during KVCache registration. To enable this feature, you need to set `--enable-cumem-allocator` or `--enable-sleep-mode` flags, and set `UCX_CUDA_IPC_ENABLE_MNNVL: 'y'` env. Otherwise, NIXL can only use RDMA/TCP for cross-node KVCache transfers.
GB-series GPUs support multi-node NVLink. NIXL supports this capability, but KVCache must be registered as VMM during KVCache registration. To enable this feature, you need to set the `--enable-cumem-allocator` flag, and set `UCX_CUDA_IPC_ENABLE_MNNVL: 'y'` env. Otherwise, NIXL can only use RDMA/TCP for cross-node KVCache transfers.
## Experimental Feature