[DCP] Support Decode Context Parallel (DCP) for GQA with FlashAttention (#24864)
Signed-off-by: yuanyongjie.yyj <yuanyongjie.yyj@antgroup.com> Signed-off-by: FENP <32334296+FENP@users.noreply.github.com> Signed-off-by: Jaya Yuan <yuanyongjie.yyj@antgroup.com>
This commit is contained in:
@@ -1523,6 +1523,7 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
|
||||
use_sliding_window=use_sliding_window,
|
||||
use_local_attention=use_local_attention,
|
||||
num_sms=self.num_sms,
|
||||
dcp_world_size=self.dcp_world_size,
|
||||
)
|
||||
return common_prefix_len if use_cascade else 0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user