DataParallelPlugin

class pytorch_lightning.plugins.training_type.DataParallelPlugin(parallel_devices)

Bases: pytorch_lightning.plugins.training_type.parallel.ParallelPlugin

Implements data-parallel training in a single process, i.e., the model gets replicated to each device and each gets a split of the data.

barrier(*args, **kwargs)

Forces all possibly joined processes to wait for each other

broadcast(obj, src=0)

Broadcasts an object to all processes

Return type

object

model_to_device()

Moves the model to the correct device

reduce(collection, *args, **kwargs)

Reduces a collection of tensors from all processes. It can be applied to just a single tensor.

Parameters

• collection (Union[Metric, Tensor, Number, Mapping[str, Union[Metric, Tensor, Number]]]) – The collection of tensors to sync and reduce.

• *args – ignored for DP

• **kwargs – ignored for DP

Return type

Union[Metric, Tensor, Number, Mapping[str, Union[Metric, Tensor, Number]]]

Returns

Reduced tensor values or the same value if it was not or did not contain a tensor.

reduce_boolean_decision(decision)

Reduce the early stopping decision across all processes

Return type

bool

setup(model)

Called by the accelerator to finish setup.

property root_device

Returns the root device