Chore: Fix minor doc sentence, grammar, quote errors (#40469)

Signed-off-by: Ashwin Phadke <23502062+ashwin-phadke@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Ashwin Phadke
2026-06-24 18:58:24 +00:00
committed by GitHub
co-authored by Harry Mellor
parent 3c43237233
commit d511b5bae9
3 changed files with 4 additions and 4 deletions
+1 -1
View File
@@ -159,7 +159,7 @@ For simplicity, we assume `block_size=1` in this section.
### High level idea
The block pool uses a dict similar to `tuple(block_hash, group_id) -> block` to catch the full blocks. That means the same tokens of different groups are cached and evicted independently.
The block pool uses a dict similar to `tuple(block_hash, group_id) -> block` to cache the full blocks. That means the same tokens of different groups are cached and evicted independently.
When a new request comes in, we check the cache hit prefix of each group, and return the intersection of these groups as the cached prefix of the request. See below for the detailed algorithm for checking the cache hit of one group & performing the intersection.
+2 -2
View File
@@ -52,7 +52,7 @@ __device__ void paged_attention_kernel(
)
```
There are also a list of template arguments above the function
There is also a list of template arguments above the function
signature that are determined during compilation time. `scalar_t`
represents the data type of the query, key, and value data elements,
such as FP16. `HEAD_SIZE` indicates the number of elements in each
@@ -178,7 +178,7 @@ const scalar_t* k_ptr = k_cache + physical_block_number * kv_block_stride
+ physical_block_offset * x;
```
Unlike to `q_ptr`, `k_ptr` in each thread will point to different
Unlike `q_ptr`, `k_ptr` in each thread will point to different
key token at different iterations. As shown above, that `k_ptr`
points to key token data based on `k_cache` at assigned block,
assigned head and assigned token.
+1 -1
View File
@@ -156,7 +156,7 @@ class EngineCoreEventType(enum.IntEnum):
class EngineCoreEvent(msgspec.Struct):
"""A timestamped engine core event associated with a request.
The timestamp is a monotonic timestamps and is used for by the engine
The timestamp is a monotonic timestamp and is used by the engine
frontend to calculate intervals between engine core events. These
timestamps should not be compared with timestamps from other processes.
"""