[Bugfix][KV Connector] Disable Mooncake TP put-striding when DCP > 1 (#45371)
Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Jingyi Yang <girasoleyang@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Jingyi Yang
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
35e4dd4a69
commit
41dcf49ca5
@@ -972,7 +972,13 @@ class MooncakeStoreWorker:
|
||||
else:
|
||||
self.num_kv_head = model_config.get_total_num_kv_heads()
|
||||
|
||||
if self.num_kv_head < self.tp_size:
|
||||
if self.num_kv_head < self.tp_size and self.dcp_size <= 1:
|
||||
# Dedup: TP ranks holding the same KV heads stripe PUTs across
|
||||
# one shared key namespace. DCP splits the TP group, so with
|
||||
# DCP>1 those ranks have different `@dcpN` namespaces and
|
||||
# striping would leave keys unwritten (OBJECT_NOT_FOUND on
|
||||
# GET). PCP is outer to TP (pcp_rank is constant within a TP
|
||||
# group), so it needs no guard.
|
||||
self.put_step = self.tp_size // self.num_kv_head
|
||||
self.head_or_tp_rank = self.tp_rank // self.put_step
|
||||
else:
|
||||
|
||||
Reference in New Issue
Block a user