[Mamba] RecoverSSM align: keep a boundary-ending state in its block
A speculative window whose accepted tokens end exactly on a recurrent block boundary selected the following block-table column for the committed state (commit plan) and recorded it as the running column (postprocess). That column can still be unallocated, so the next step reads an uninitialized page. Select the block that holds the last committed token, matching the (n - 1) // block convention used elsewhere in align mode. Applies to Kimi-K3 KDA and to the Qwen GDN/PLE RecoverSSM paths that share the plan. Fix and test from local-inference-lab/vllm a02f585d (Martin Vit, root cause reported by astra); the GDN/PLE align tests now expect the same convention. Signed-off-by: Jürgen Schmied <juergenschmied70@gmail.com> Co-authored-by: Martin Vit <martin@voipmonitor.org> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
co-authored by
Martin Vit
Claude
parent
ed2a9665ea
commit
5567cc1b25
@@ -78,8 +78,12 @@ def prepare_commit_plan_kernel(
|
||||
tl.int32
|
||||
)
|
||||
final_num_computed = num_computed + commit_len
|
||||
# The committed state belongs to the block holding the last committed
|
||||
# token. A window ending exactly on a boundary stays in that block; the
|
||||
# next block may still be unallocated.
|
||||
final_state_col = tl.minimum(
|
||||
final_num_computed // mamba_block_size, block_table_width - 1
|
||||
tl.maximum(final_num_computed - 1, 0) // mamba_block_size,
|
||||
block_table_width - 1,
|
||||
)
|
||||
final_state_idx = tl.load(
|
||||
block_table_ptr
|
||||
|
||||
Reference in New Issue
Block a user