DataParallelPlugin

class pytorch_lightning.plugins.training_type.DataParallelPlugin(parallel_devices)

Bases: pytorch_lightning.plugins.training_type.parallel.ParallelPlugin

Implements data-parallel training in a single process, i.e., the model gets replicated to each device and each gets a split of the data.

barrier(*args, **kwargs)

Forces all possibly joined processes to wait for each other

broadcast(obj, src=0)

Broadcasts an object to all processes

Return type

object

model_to_device()

Moves the model to the correct device

reduce(tensor, *args, **kwargs)

Reduces a tensor from all parallel processes to one aggregated tensor.

Parameters

• tensor – the tensor to sync and reduce

• *args – ignored for DP

• **kwargs – ignored for DP

Returns

reduced value, except when the input was not a tensor the output remains is unchanged

reduce_boolean_decision(decision)

Reduce the early stopping decision across all processes

Return type

bool

setup(model)

Called by the accelerator to finish setup.

property root_device

Returns the root device