Shortcuts

DDPSpawnStrategy

class pytorch_lightning.strategies.DDPSpawnStrategy(accelerator=None, parallel_devices=None, cluster_environment=None, checkpoint_io=None, precision_plugin=None, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, **kwargs)[source]

Bases: pytorch_lightning.strategies.parallel.ParallelStrategy

Spawns processes using the torch.multiprocessing.spawn() method and joins processes after training finishes.

add_to_queue(trainer, queue)[source]

Appends the trainer.callback_metrics dictionary to the given queue. To avoid issues with memory sharing, we cast the data to numpy.

Parameters
  • trainer (Trainer) – reference to the Trainer.

  • queue (_FakeQueue) – the instance of the queue to append the data.

Return type

None

barrier(*args, **kwargs)[source]

Synchronizes all processes which blocks processes until the whole group enters this function.

Parameters

name – an optional name to pass into barrier.

Return type

None

broadcast(obj, src=0)[source]

Broadcasts an object to all processes.

Parameters
  • obj (object) – the object to broadcast

  • src (int) – source rank

Return type

object

get_from_queue(trainer, queue)[source]

Retrieve the trainer.callback_metrics dictionary from the given queue. To preserve consistency, we cast back the data to torch.Tensor.

Parameters
  • trainer (Trainer) – reference to the Trainer.

  • queue (_FakeQueue) – the instance of the queue from where to get the data.

Return type

None

model_to_device()[source]

Moves the model to the correct device.

pre_backward(closure_loss)[source]

Run before precision plugin executes backward.

Return type

None

predict_step(*args, **kwargs)[source]

The actual predict step.

See predict_step() for more details

Return type

Union[Tensor, Dict[str, Any]]

reduce(tensor, group=None, reduce_op='mean')[source]

Reduces a tensor from several distributed processes to one aggregated tensor.

Parameters
  • tensor – the tensor to sync and reduce

  • group (Optional[Any]) – the process group to gather results from. Defaults to all processes (world)

  • reduce_op (Union[ReduceOp, str]) – the reduction operation. Defaults to ‘mean’/’avg’. Can also be a string ‘sum’ to calculate the sum during reduction.

Return type

Tensor

Returns

reduced value, except when the input was not a tensor the output remains is unchanged

setup(trainer)[source]

Setup plugins for the trainer fit and creates optimizers.

Parameters

trainer (Trainer) – the trainer instance

Return type

None

spawn(function, *args, **kwargs)[source]

Spawn processes that run the given function.

Parameters
  • function (Callable) – The function to spawn processes from.

  • *args – Optional positional arguments that will be passed to the function in addition to the process index. These arguments must be pickleable.

  • **kwargs – Optional named arguments that will be passed to the function in addition to the process index. These arguments must be pickleable.

Return type

Union[Any, _SpawnOutput, None]

Returns

The output of the function of process 0.

teardown()[source]

This method is called to teardown the training process.

It is the right place to release memory and free other resources.

Return type

None

test_step(*args, **kwargs)[source]

The actual test step.

See test_step() for more details

Return type

Union[Tensor, Dict[str, Any], None]

training_step(*args, **kwargs)[source]

The actual training step.

See training_step() for more details

Return type

Union[Tensor, Dict[str, Any]]

validation_step(*args, **kwargs)[source]

The actual validation step.

See validation_step() for more details

Return type

Union[Tensor, Dict[str, Any], None]

property root_device

Return the root device.

Read the Docs v: latest
Versions
latest
stable
1.5.9
1.5.8
1.5.7
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.10
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.8
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.8
1.0.7
1.0.6
1.0.5
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.10.0
0.9.0
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.0
0.5.3
0.4.9
ipynb-update
docs-search
Downloads
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.