[KV Cache] GLM-5.3-Flash: use the generic packed KV layout; CircularBufferSpec tail with speculative ring slots (#57169)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Signed-off-by: Dakai An <dakaian108@gmail.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
This commit is contained in:
co-authored by
Yifan Qiao
OpenAI Codex
OpenAI Codex
Lucas Wilkinson
Claude Opus 5.5
Lucas Wilkinson
parent
3766a67a32
commit
02e55d2af3
@@ -95,11 +95,9 @@ def prepare_dcp_dummy_context_metadata(
|
||||
max_valid_block_id = kv_cache_config.num_blocks - 1
|
||||
assert max_valid_block_id > 0
|
||||
for blk_table in input_batch.block_table.block_tables:
|
||||
max_row_blocks = (
|
||||
blk_table.max_num_blocks_per_req // blk_table.blocks_per_kv_block
|
||||
)
|
||||
block_ids = [
|
||||
(block_idx % max_valid_block_id) + 1 for block_idx in range(max_row_blocks)
|
||||
(block_idx % max_valid_block_id) + 1
|
||||
for block_idx in range(blk_table.max_num_blocks_per_req)
|
||||
]
|
||||
for req_idx in range(num_reqs):
|
||||
blk_table.add_row(block_ids, req_idx)
|
||||
|
||||
Reference in New Issue
Block a user