[Feature] Support sequence parallel without the need for DP, 1.9%~5.0% E2E Throughput Improvement (#47070)
Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: gcanlin <canlinguosdu@gmail.com> Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
This commit is contained in:
@@ -175,10 +175,11 @@ class DeviceCommunicatorBase:
|
||||
|
||||
config = get_current_vllm_config_or_none()
|
||||
if config is not None:
|
||||
# as long as we use data parallel (coupled data parallel
|
||||
# where all data parallel ranks execute forward together),
|
||||
# we initialize the all2all manager used in expert parallel.
|
||||
use_ep = config.parallel_config.data_parallel_size > 1
|
||||
# initialize the all2all manager for DP or sequence-parallel EP.
|
||||
use_ep = (
|
||||
config.parallel_config.data_parallel_size > 1
|
||||
or config.parallel_config.use_sequence_parallel_moe
|
||||
)
|
||||
all2all_backend = config.parallel_config.all2all_backend
|
||||
|
||||
self.is_ep_communicator = unique_name.split(":")[0] == "ep"
|
||||
|
||||
Reference in New Issue
Block a user