[Bugfix][MoRIIO] Exclude synchronous READ destinations from KV zeroing (#59164)

Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Signed-off-by: whx-sjtu <xiaowang990929@gmail.com>
Signed-off-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com>
Co-authored-by: Yichao Zhu <Yichao.Zhu@amd.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai>
This commit is contained in:
Hexiang Wang
2026-10-07 16:35:56 +08:00
committed by GitHub
co-authored by Yichao Zhu Codex Yifan Qiao
parent 42ecc8eebe
commit df417f780a
3 changed files with 64 additions and 24 deletions
@@ -583,7 +583,11 @@ class KVConnectorBase_V1(ABC):
"""Build the connector metadata for this step.
This function should NOT modify fields in the scheduler_output.
Also, calling this function will reset the state of the connector.
FIXME: one exception:
synchronous READ connectors may remove attention blocks they fully
overwrite this step from new_block_ids_to_zero. They must complete
those loads before the blocks are used and fail the step if a load fails.
Calling this function will reset the state of the connector.
Args:
scheduler_output (SchedulerOutput): the scheduler output object.
@@ -1232,6 +1232,24 @@ class MoRIIOConnectorScheduler:
local_block_ids=block_ids,
kv_transfer_params=kv_params,
)
if (
self.mode == MoRIIOMode.READ
and self._has_mamba
and scheduler_output.new_block_ids_to_zero
):
# Hybrid models zero recycled attention pages that held Mamba state.
# Host-submitted READs overwrite these pages and can race zeroing.
# Hybrid READ metadata puts the aligned attention pages first.
read_dst_block_ids = {
b
for _, block_ids in self._reqs_need_recv.values()
for b in block_ids[0]
}
scheduler_output.new_block_ids_to_zero = [
b
for b in scheduler_output.new_block_ids_to_zero
if b not in read_dst_block_ids
]
for req_id, (req, block_ids) in self._reqs_need_save.items():
kv_params = self._req_kv_params.get(req_id, req.kv_transfer_params or {})