API References¶

Accelerator API¶

`datamodule`	LightningDataModule for loading DataLoaders with ease.
`decorators`
`hooks`	Various hooks to be used in the Lightning code.
`lightning`	The LightningModule - an nn.Module with many additional features.

`base`	Abstract base class used to build new callbacks.
`early_stopping`	Early Stopping
`gpu_stats_monitor`	GPU Stats Monitor
`gradient_accumulation_scheduler`	Gradient Accumulator
`lr_monitor`	Learning Rate Monitor
`model_checkpoint`	Model Checkpointing
`progress`	Progress Bars

`base`	Abstract base class used to build new loggers.
`comet`	Comet Logger
`csv_logs`	CSV logger
`mlflow`	MLflow Logger
`neptune`	Neptune Logger
`tensorboard`	TensorBoard Logger
`test_tube`	Test Tube Logger
`wandb`	Weights and Biases Logger

`Loop`	Basic Loops interface.
`DataLoaderLoop`	Base class to loop over all dataloaders.

`FitLoop`	This Loop iterates over the epochs to run the training.
`TrainingEpochLoop`	Runs over all batches in a dataloader (one epoch).
`TrainingBatchLoop`	Runs over a single batch of data.
`OptimizerLoop`	Runs over a sequence of optimizers.
`ManualOptimization`	A special loop implementing what is known in Lightning as Manual Optimization where the optimization happens entirely in the `training_step()` and therefore the user is responsible for back-propagating gradients and making calls to the optimizers.

`EvaluationLoop`	Loops over all dataloaders for evaluation.
`EvaluationEpochLoop`	This is the loop performing the evaluation.

`PredictionLoop`	Loop to run over dataloaders for prediction.
`PredictionEpochLoop`	Loop performing prediction on arbitrary sequentially used dataloaders.

`TrainingTypePlugin`	Base class for all training type plugins that change the behaviour of the training, validation and test- loop.
`SingleDevicePlugin`	Plugin that handles communication on a single device.
`ParallelPlugin`	Plugin for training with multiple processes in parallel.
`DataParallelPlugin`	Implements data-parallel training in a single process, i.e., the model gets replicated to each device and each gets a split of the data.
`DDPPlugin`	Plugin for multi-process single-device training on one or multiple nodes.
`DDP2Plugin`	DDP2 behaves like DP in one node, but synchronization across nodes behaves like in DDP.
`DDPShardedPlugin`	Optimizer and gradient sharded training provided by FairScale.
`DDPSpawnShardedPlugin`	Optimizer sharded training provided by FairScale.
`DDPSpawnPlugin`	Spawns processes using the `torch.multiprocessing.spawn()` method and joins processes after training finishes.
`DeepSpeedPlugin`	Provides capabilities to run training using the DeepSpeed library, with training optimizations for large billion parameter models.
`HorovodPlugin`	Plugin for Horovod distributed training integration.
`SingleTPUPlugin`	Plugin for training on a single TPU device.
`TPUSpawnPlugin`	Plugin for training multiple TPU devices using the `torch.multiprocessing.spawn()` method.

`PrecisionPlugin`	Base class for all plugins handling the precision-specific parts of the training.
`MixedPrecisionPlugin`	Base Class for mixed precision.
`NativeMixedPrecisionPlugin`	Plugin for Native Mixed Precision (AMP) training with `torch.autocast`.
`ShardedNativeMixedPrecisionPlugin`	Native AMP for Sharded Training.
`ApexMixedPrecisionPlugin`	Mixed Precision Plugin based on Nvidia/Apex (https://github.com/NVIDIA/apex)
`DeepSpeedPrecisionPlugin`	Precision plugin for DeepSpeed integration.
`TPUPrecisionPlugin`
`TPUBf16PrecisionPlugin`	Plugin that enables bfloats on TPUs.
`DoublePrecisionPlugin`	Plugin for training with double (`torch.float64`) precision.
`FullyShardedNativeMixedPrecisionPlugin`	Native AMP for Fully Sharded Training.
`IPUPrecisionPlugin`

`ClusterEnvironment`	Specification of a cluster environment.
`LightningEnvironment`	The default environment used by Lightning for a single node or free cluster (not managed).
`LSFEnvironment`	An environment for running on clusters managed by the LSF resource manager.
`TorchElasticEnvironment`	Environment for fault-tolerant and elastic training with torchelastic
`KubeflowEnvironment`	Environment for distributed training using the PyTorchJob operator from Kubeflow
`SLURMEnvironment`	Cluster environment for training on a cluster managed by SLURM.

`CheckpointIO`	Interface to save/load checkpoints as they are saved through the `TrainingTypePlugin`.
`TorchCheckpointIO`	CheckpointIO that utilizes `torch.save()` and `torch.load()` to save and load checkpoints respectively, common for most use cases.
`XLACheckpointIO`	CheckpointIO that utilizes `xm.save()` to save checkpoints for TPU training strategies.

`AbstractProfiler`	Specification of a profiler.
`AdvancedProfiler`	This profiler uses Python’s cProfiler to record more detailed information about time spent in each function call recorded during a given action.
`BaseProfiler`	If you wish to write a custom profiler, you should inherit from this class.
`PassThroughProfiler`	This class should be used when you don’t want the (small) overhead of profiling.
`PyTorchProfiler`	This profiler uses PyTorch’s Autograd Profiler and lets you inspect the cost of.
`SimpleProfiler`	This profiler simply records the duration of actions (in seconds) and reports the mean duration of each action and the total time spent over the entire training run.
`XLAProfiler`	This Profiler will help you debug and optimize training workload performance for your models using Cloud TPU performance tools.

Trainer to automate the training.

Lite accelerates your PyTorch training or inference code with minimal changes required.

Tuner class to tune your model.

`cli`
`argparse`
`seed`	Helper functions to help with reproducibility of models.