Shortcuts

SingleDevicePlugin

class pytorch_lightning.plugins.training_type.SingleDevicePlugin(device)[source]

Bases: pytorch_lightning.plugins.training_type.training_type_plugin.TrainingTypePlugin

Plugin that handles communication on a single device.

all_gather(tensor, group=None, sync_grads=False)[source]

Perform a all_gather on all processes

Return type

Tensor

barrier(*args, **kwargs)[source]

Forces all possibly joined processes to wait for each other

Return type

None

broadcast(obj, src=0)[source]

Broadcasts an object to all processes

Return type

object

model_to_device()[source]

Moves the model to the correct device

Return type

None

reduce(tensor, *args, **kwargs)[source]

Reduces a tensor from several distributed processes to one aggregated tensor. As this plugin only operates with a single device, the reduction is simply the identity.

Parameters
  • tensor (Union[Any, Tensor]) – the tensor to sync and reduce

  • *args – ignored

  • **kwargs – ignored

Return type

Union[Any, Tensor]

Returns

the unmodified input as reduction is not needed for single process operation

setup(model)[source]

Called by the accelerator to finish setup.

Return type

Module

teardown()[source]

This method is called to teardown the training process. It is the right place to release memory and free other resources.

Return type

None

property is_global_zero: bool

Whether the current process is the rank zero process not only on the local node, but for all nodes.

Return type

bool

property on_gpu: bool

Returns whether the current process is done on GPU

Return type

bool

property on_tpu: bool

Returns whether the current process is done on TPU

Return type

bool

property root_device: torch.device

Returns the root device

Return type

device