Get Started
Blog
Docs
PyTorch Lightning TorchMetrics Lightning Flash Lightning Transformers Lightning Bolts
GitHub
Train on the cloud with Lightning

Table of Contents

1.8.6

Get Started

Lightning in 15 minutes
Installation

Level Up

Basic skills
Intermediate skills
Advanced skills
Expert skills

Core API

LightningModule
Trainer

API Reference

accelerators
callbacks
cli
core
lightninglite
loggers
profiler
trainer
strategies
tuner
utilities

Common Workflows

Avoid overfitting
Build a Model
Configure hyperparameters from the CLI
Customize the progress bar
Deploy models into production
Effective Training Techniques
Eliminate config boilerplate
Find bottlenecks in your code
Manage experiments
Organize existing PyTorch into Lightning
Run on an on-prem cluster
Save and load model progress
Save memory with half-precision
Training over the internet
Train 1 trillion+ parameter models
Train on the cloud
Train on single or multiple GPUs
Train on single or multiple HPUs
Train on single or multiple IPUs
Train on single or multiple TPUs
Train on MPS
Use a pretrained model
Inject Custom Data Iterables
Use a pure PyTorch training loop

Glossary

Hands-on Examples

Tutorial 1: Introduction to PyTorch
Tutorial 2: Activation Functions
Tutorial 3: Initialization and Optimization
Tutorial 4: Inception, ResNet and DenseNet
Tutorial 5: Transformers and Multi-Head Attention
Tutorial 6: Basics of Graph Neural Networks
Tutorial 7: Deep Energy-Based Generative Models
Tutorial 8: Deep Autoencoders
Tutorial 9: Normalizing Flows for Image Modeling
Tutorial 10: Autoregressive Image Modeling
Tutorial 11: Vision Transformers
Tutorial 12: Meta-Learning - Learning to Learn
Tutorial 13: Self-Supervised Contrastive Learning with SimCLR
GPU and batched data augmentation with Kornia and PyTorch-Lightning
Barlow Twins Tutorial
PyTorch Lightning Basic GAN Tutorial
PyTorch Lightning CIFAR10 ~94% Baseline Tutorial
PyTorch Lightning DataModules
Fine-Tuning Scheduler
Introduction to Pytorch Lightning
TPU training with PyTorch Lightning
How to train a Deep Q Network
Finetune Transformers Models with PyTorch Lightning
Multi-agent Reinforcement Learning With WarpDrive
PyTorch Lightning 101 class
From PyTorch to PyTorch Lightning [Blog]
From PyTorch to PyTorch Lightning [Video]

Community

Contributor Covenant Code of Conduct
Contributing
How to Become a Core Contributor
Lightning Governance
Versioning Policy
Changelog

Docs >
accelerators >
optimizer
Edit on GitHub

Shortcuts

optimizer¶

Functions

optimizer_to_device

rtype: None

optimizers_to_device

rtype: None

Next

Previous

© Copyright Copyright (c) 2018-2023, Lightning AI et al... Revision caa3329b.

Built with Sphinx using a theme provided by Read the Docs.

optimizer

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. Read PyTorch Lightning's Privacy Policy.

Get Started
Blog
Ecosystem

PyTorch Lightning
TorchMetrics
Lightning Flash
Lightning Transformers
Lightning Bolts

Github
Lightning.ai