[Bugfix] release KV blocks for skipped P-ranks to prevent invalid KV errors and timeouts when P_tp > D_tp and MLA (#40449)
Signed-off-by: yangruize <yangruize7@163.com> Co-authored-by: Roger Wang <hey@rogerw.io>
This commit is contained in:
@@ -1971,7 +1971,7 @@ class NixlConnectorWorker:
|
||||
if self.use_mla and tp_ratio < 0:
|
||||
# ..but we still need to notify the other remote ranks that we
|
||||
# have the blocks we need so they can update the request state.
|
||||
notif_id = f"{req_id}:{self.world_size}".encode()
|
||||
notif_id = f"{meta.remote.request_id}:{self.world_size}".encode()
|
||||
remote_agents = self._remote_agents[meta.remote.engine_id]
|
||||
for rank_to_notify, agent in remote_agents.items():
|
||||
if rank_to_notify != remote_rank:
|
||||
|
||||
Reference in New Issue
Block a user