[DCP] Support Decode Context Parallel (DCP) for GQA with FlashAttention (#24864)

Signed-off-by: yuanyongjie.yyj <yuanyongjie.yyj@antgroup.com>
Signed-off-by: FENP <32334296+FENP@users.noreply.github.com>
Signed-off-by: Jaya Yuan <yuanyongjie.yyj@antgroup.com>
This commit is contained in:
Jaya Yuan
2025-10-14 13:07:50 +00:00
committed by GitHub
parent fdd32750f0
commit ea97940d6c
7 changed files with 209 additions and 33 deletions
+1
View File
@@ -1523,6 +1523,7 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
use_sliding_window=use_sliding_window,
use_local_attention=use_local_attention,
num_sms=self.num_sms,
dcp_world_size=self.dcp_world_size,
)
return common_prefix_len if use_cascade else 0