[KV Connector] Support symmetric DCP disagg for hybrid mamba models (#55531)
Signed-off-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster> Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com> Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> Co-authored-by: Wei Zhao <weizha@oci-aga-slurm-1-vscode-02.cm.cluster> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
Wei Zhao
OpenAI Codex
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
983b7e28c9
commit
7cdd9304ae
@@ -393,8 +393,15 @@ class ParallelConfig:
|
||||
in (rank i+1, block j) only after (rank i, block j) is fully occupied.
|
||||
Block_size should be greater than or equal to cp_kv_cache_interleave_size.
|
||||
Block_size should be divisible by cp_kv_cache_interleave_size.
|
||||
|
||||
When --cp-kv-cache-interleave-size is omitted (None), the interleave size
|
||||
is resolved automatically based on NIXL transfer requirements.
|
||||
Explicit settings take priority.
|
||||
"""
|
||||
|
||||
_allow_auto_resolve_cp_interleave_size: bool = True
|
||||
"""Whether NIXL may select the interleave size automatically."""
|
||||
|
||||
data_parallel_index: int = Field(init=False)
|
||||
"""Equal to the data parallel rank but not used for torch process groups
|
||||
and not overridden for dense models."""
|
||||
|
||||
Reference in New Issue
Block a user