Shortcuts

DDPSpawnPlugin

class pytorch_lightning.plugins.training_type.DDPSpawnPlugin(parallel_devices=None, num_nodes=None, cluster_environment=None, sync_batchnorm=None, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, **kwargs)[source]

Bases: pytorch_lightning.plugins.training_type.parallel.ParallelPlugin

Spawns processes using the torch.multiprocessing.spawn() method and joins processes after training finishes.

barrier(*args, **kwargs)[source]

Forces all possibly joined processes to wait for each other

broadcast(obj, src=0)[source]

Broadcasts an object to all processes

Return type

object

model_to_device()[source]

Moves the model to the correct device

post_dispatch()[source]

Hook to do something after the training/evaluation/prediction finishes.

pre_backward(closure_loss, should_accumulate, optimizer, opt_idx)[source]

Run before precision plugin executes backward

reduce(tensor, group=None, reduce_op='mean')[source]

Reduces a tensor from several distributed processes to one aggregated tensor.

Parameters
  • tensor – the tensor to sync and reduce

  • group (Optional[Any]) – the process group to gather results from. Defaults to all processes (world)

  • reduce_op (Union[ReduceOp, str, None]) – the reduction operation. Defaults to ‘mean’/’avg’. Can also be a string ‘sum’ to calculate the sum during reduction.

Returns

reduced value, except when the input was not a tensor the output remains is unchanged

setup(model)[source]

Called by the accelerator to finish setup.

property root_device

Returns the root device

Read the Docs v: latest
Versions
latest
stable
1.3.1
1.3.0
1.2.10
1.2.9_a
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.8
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.8
1.0.7
1.0.6
1.0.5
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.10.0
0.9.0
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.0
0.5.3
0.4.9
docs-robots
Downloads
pdf
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.