- class pytorch_lightning.plugins.NativeSyncBatchNorm¶
A plugin that wraps all batch normalization layers of a model with synchronization logic for multiprocessing.
This plugin has no effect in single-device operation.
Add global batchnorm for a model spread across multiple GPUs and nodes.
Override this method to synchronize batchnorm layers between specific process groups instead of the whole world.
Module) – Reference to the current LightningModule
- Return type
LightningModule with batchnorm layers synchronized within the process groups.