DDP2Plugin

class pytorch_lightning.plugins.training_type.DDP2Plugin(parallel_devices=None, num_nodes=None, cluster_environment=None, sync_batchnorm=None, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, **kwargs)

Bases: pytorch_lightning.plugins.training_type.ddp.DDPPlugin

DDP2 behaves like DP in one node, but synchronization across nodes behaves like in DDP.

model_to_device()

Moves the model to the correct device

reduce(collection, *args, **kwargs)

Reduces a collection of tensors from all processes. It can be applied to just a single tensor. In DDP2, the reduction here is only across local devices within the node.

Parameters

• collection (Union[Metric, Tensor, Number, Mapping[str, Union[Metric, Tensor, Number]]]) – The collection of tensors to sync and reduce.

• *args – ignored for DDP2

• **kwargs – ignored for DDP2

Return type

Union[Metric, Tensor, Number, Mapping[str, Union[Metric, Tensor, Number]]]

Returns

Reduced tensor values or the same value if it was not or did not contain a tensor.

setup(model)

Called by the accelerator to finish setup.

property root_device

Returns the root device