[PCP] Support prefill context parallelism with data parallelism (#57075)
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit is contained in:
co-authored by
QiuChunshuo
Claude Fable 5.1
Codex
Robert Shaw
Nick Hill
mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
parent
6697a7dd30
commit
e6dc16cebd
@@ -364,7 +364,7 @@ class DraftModelSpeculator(BaseSpeculator):
|
||||
kv_cache_config=self.kv_cache_config,
|
||||
causal=causal,
|
||||
seq_lens_cpu_upper_bound=draft_seq_lens_cpu_upper_bound,
|
||||
is_prefilling=self.draft_is_prefilling[:num_reqs],
|
||||
is_prefilling=self.draft_is_prefilling[:num_reqs_padded],
|
||||
)
|
||||
return attn_metadata
|
||||
|
||||
|
||||
Reference in New Issue
Block a user