[Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA (#40449)

Signed-off-by: yangruize <yangruize7@163.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
yangrz
2026-04-28 11:38:43 -07:00
committed by GitHub
co-authored by Roger Wang
parent 0899f436aa
commit de3fe8dc62
2 changed files with 120 additions and 1 deletions
@@ -1971,7 +1971,7 @@ class NixlConnectorWorker:
if self.use_mla and tp_ratio < 0:
# ..but we still need to notify the other remote ranks that we
# have the blocks we need so they can update the request state.
notif_id = f"{req_id}:{self.world_size}".encode()
notif_id = f"{meta.remote.request_id}:{self.world_size}".encode()
remote_agents = self._remote_agents[meta.remote.engine_id]
for rank_to_notify, agent in remote_agents.items():
if rank_to_notify != remote_rank: