DDPSpawnPlugin¶
-
class
pytorch_lightning.plugins.training_type.
DDPSpawnPlugin
(parallel_devices=None, num_nodes=1, cluster_environment=None, sync_batchnorm=False, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, **kwargs)[source]¶ Bases:
pytorch_lightning.plugins.training_type.parallel.ParallelPlugin
Spawns processes using the
torch.multiprocessing.spawn()
method and joins processes after training finishes.-
barrier
(*args, **kwargs)[source]¶ Forces all possibly joined processes to wait for each other
- Return type
-
pre_backward
(closure_loss, should_accumulate, optimizer, opt_idx)[source]¶ Run before precision plugin executes backward
-
reduce
(tensor, group=None, reduce_op='mean')[source]¶ Reduces a tensor from several distributed processes to one aggregated tensor.
- Parameters
- Returns
reduced value, except when the input was not a tensor the output remains is unchanged
-
property
root_device
¶ Returns the root device
-