Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Chao Lei <leichao139636@163.com> Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Chao Lei <leichao139636@163.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <noreply@openai.com>
This commit is contained in:
co-authored by
Chao Lei
Claude Fable 5
Matthew Bonanni
OpenAI Codex
OpenAI Codex
parent
ebe1dec2da
commit
e19a3e172e
@@ -1435,6 +1435,12 @@ class KVCacheConfig:
|
||||
hisparse_host_num_blocks: int | None = None
|
||||
"""Capacity of the dedicated HiSparse host-block manager, when enabled."""
|
||||
|
||||
hisparse_host_block_stride: int | None = None
|
||||
"""Physical bytes between consecutive HiSparse host blocks."""
|
||||
|
||||
hisparse_shared_host_pool: bool = False
|
||||
"""Whether local TP ranks share one physical HiSparse host pool."""
|
||||
|
||||
@cached_property
|
||||
def transfer_group_ids(self) -> tuple[int, ...]:
|
||||
"""IDs of cache groups that participate in external KV transfer."""
|
||||
|
||||
Reference in New Issue
Block a user