[HMA] [KVEvent] Enable GPU-side KV events for HMA (#37688)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com> Co-authored-by: Or Ozeri <or@ozery.com>
This commit is contained in:
co-authored by
Or Ozeri
parent
17e787a779
commit
cc07dad789
@@ -43,10 +43,13 @@ class BlockStored(KVCacheEvent):
|
||||
prompt embeddings data, etc. for that specific block.
|
||||
"""
|
||||
|
||||
group_idx: int | None = None
|
||||
|
||||
|
||||
class BlockRemoved(KVCacheEvent):
|
||||
block_hashes: list[ExternalBlockHash]
|
||||
medium: str | None
|
||||
group_idx: int | None = None
|
||||
|
||||
|
||||
class AllBlocksCleared(KVCacheEvent):
|
||||
|
||||
Reference in New Issue
Block a user