[KV Connector][Mooncake] Add tenant ID support to MooncakeStoreConnector (#48069)
Signed-off-by: lzw <2994587066@qq.com> Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
@@ -61,6 +61,9 @@ Create a JSON configuration file (e.g., `mooncake_config.json`):
|
||||
large prefills do not exceed the owner's SSD-write budget. Set this together
|
||||
with the matching `--enable_offload=true` flag on `mooncake_master` and on
|
||||
the external `mooncake_client` (if any).
|
||||
- `tenant_id`: Optional Mooncake tenant namespace. Producers and consumers
|
||||
that should share store data must use the same tenant id. Default:
|
||||
`"default"`.
|
||||
|
||||
Set the config path via environment variable:
|
||||
|
||||
@@ -181,6 +184,26 @@ Mooncake environment variables (`MOONCAKE_OFFLOAD_FILE_STORAGE_PATH`,
|
||||
`MOONCAKE_OFFLOAD_TOTAL_SIZE_LIMIT_BYTES`, etc.). Those are independent of
|
||||
the vLLM JSON config.
|
||||
|
||||
### Tenant Isolation
|
||||
|
||||
Set `tenant_id` in the Mooncake JSON config when different vLLM deployments should use separate Mooncake tenant namespaces:
|
||||
|
||||
```json
|
||||
{
|
||||
"mode": "embedded",
|
||||
"metadata_server": "P2PHANDSHAKE",
|
||||
"master_server_address": "127.0.0.1:50051",
|
||||
"global_segment_size": "80GB",
|
||||
"local_buffer_size": "4GB",
|
||||
"protocol": "rdma",
|
||||
"device_name": "",
|
||||
"enable_offload": false,
|
||||
"tenant_id": "tenant-a"
|
||||
}
|
||||
```
|
||||
|
||||
Strict isolation requires a Mooncake master started with `--enable_multi_tenants=true` and a tenant quota policy that registers each tenant. Non-default `tenant_id` also requires a Mooncake version whose `MooncakeDistributedStore.setup()` accepts the `tenant_id` parameter. In `standalone-store` mode, start the external `mooncake_client` with the matching tenant id because that process owns the real store client.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
|
||||
Reference in New Issue
Block a user