[Bugfix] Give each data-parallel engine its own global RNG streams (#59788)
Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
877ddcc456
commit
d09ff77637
@@ -1020,7 +1020,9 @@ class Worker(WorkerBase):
|
||||
|
||||
# Reset the seed to ensure that the random state is not affected by
|
||||
# the model initialization and profiling.
|
||||
set_random_seed(self.model_config.seed)
|
||||
set_random_seed(
|
||||
self.model_config.seed, self.parallel_config.data_parallel_index
|
||||
)
|
||||
|
||||
# Eagerly trigger inductor's once-per-process lazy inits during
|
||||
# warmup (rather than on a later compile cache-miss at runtime).
|
||||
|
||||
Reference in New Issue
Block a user