[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Signed-off-by: Dakai An <dakaian108@gmail.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
This commit is contained in:
co-authored by
Yifan Qiao
OpenAI Codex
OpenAI Codex
Lucas Wilkinson
Claude Opus 5.5
Lucas Wilkinson
parent
3766a67a32
commit
02e55d2af3
@@ -95,7 +95,9 @@ class CompressedSlotMappingKernel(
|
||||
block_table_ptr + batch_idx * block_table_stride + block_ids,
|
||||
mask=mask & is_valid,
|
||||
)
|
||||
slot_ids = block_numbers * block_size + pos_after_compress % block_size
|
||||
slot_ids = block_numbers.to(tl.int64) * block_size + (
|
||||
pos_after_compress % block_size
|
||||
)
|
||||
|
||||
# NOTE
|
||||
slot_ids = tl.where(is_valid, slot_ids, PAD_ID)
|
||||
|
||||
Reference in New Issue
Block a user