Shortcuts

Lightning Transformers

Lightning Transformers offers a flexible interface for training and fine-tuning SOTA Transformer models using the PyTorch Lightning Trainer.

pip install lightning-transformers

In Lightning Transformers, we offer the following benefits:

  • Powered by PyTorch Lightning - Accelerators, custom Callbacks, Loggers, and high performance scaling with minimal changes.

  • Backed by HuggingFace Transformers models and datasets, spanning multiple modalities and tasks within NLP/Audio and Vision.

  • Task Abstraction for Rapid Research & Experimentation - Build your own custom transformer tasks across all modalities with little friction.

  • Powerful config composition backed by Hydra - simply swap out models, optimizers, schedulers task, and many more configurations without touching the code.

  • Seamless Memory and Speed Optimizations - Out-of-the-box training optimizations such as DeepSpeed ZeRO or FairScale Sharded Training with no code changes.


Using Lightning-Transformers

Lightning Transformers has a collection of tasks for common NLP problems such as language_modeling, translation and more. To use, simply:

  1. Pick a task to train (passed to train.py as task=)

  2. Pick a dataset (passed to train.py as dataset=)

  3. Customize the backbone, optimizer, or any component within the config

  4. Add any Lightning supported parameters and optimizations

python train.py \
    task=<TASK> \
    dataset=<DATASET>
    backbone.pretrained_model_name_or_path=<BACKBONE> # Optionally change the HF backbone
    optimizer=<OPTIMIZER> # Optionally specify optimizer (Default AdamW)
    trainer.<ANY_TRAINER_FLAGS> # Optionally specify Lightning trainer arguments

To learn more about Lightning Transformers, please refer to the Lightning Transformers documentation.


© Copyright Copyright (c) 2018-2022, Lightning AI et al... Revision be581598.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: stable
Versions
latest
stable
1.7.1
1.7.0
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.10
1.5.9
1.5.8
1.5.7
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.10
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.8
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.8
1.0.7
1.0.6
1.0.5
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.10.0
0.9.0
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.0
0.5.3
0.4.9
future-structure
Downloads
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.