DDPShardedPlugin¶

class pytorch_lightning.plugins.training_type.DDPShardedPlugin(*args, **kwargs)[source]¶

Optimizer and gradient sharded training provided by FairScale.

block_backward_sync()[source]¶

Blocks syncing gradients behaviour on backwards pass.

This is useful for skipping sync when accumulating gradients, reducing communication overhead Returns: context manager with sync behaviour off

pre_backward(closure_loss)[source]¶

Run before precision plugin executes backward.

property lightning_module: pytorch_lightning.core.lightning.LightningModule¶

Returns the pure LightningModule without potential wrappers.