[Doc] Fix cross-reference warnings (#25058)

Signed-off-by: Punit Vara <punitvara@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Punitvara
2025-09-18 02:05:16 -07:00
committed by GitHub
co-authored by Harry Mellor
parent aa3f105c59
commit 05b044e698
6 changed files with 18 additions and 14 deletions
+3 -2
View File
@@ -24,8 +24,9 @@ class KVCacheBlocks:
"""
blocks: tuple[list[KVCacheBlock], ...]
"""
blocks[i][j] refers to the i-th kv_cache_group and the j-th block of tokens.
We don't use block of tokens as the outer dimension because it assumes all
`blocks[i][j]` refers to the i-th kv_cache_group
and the j-th block of tokens.We don't use block of
tokens as the outer dimension because it assumes all
kv_cache_groups have the same number of blocks, which is true for now but
will be broken if we want to give different block_size to different
kv_cache_groups in the future.