Shortcuts

Accelerator: HPU training

Audience: Gaudi chip users looking to save memory and scale models with mixed-precision training.


Enable Mixed Precision

Lightning also allows mixed precision training with HPUs. By default, HPU training will use 32-bit precision. To enable mixed precision, set the precision flag.

trainer = Trainer(devices=1, accelerator="hpu", precision=16)

Customize Mixed Precision

Internally, HPUPrecisionPlugin uses the Habana Mixed Precision (HMP) package to enable mixed precision training.

You can execute the ops in FP32 or BF16 precision. The HMP package modifies the Python operators to add the appropriate cast operations for the arguments before execution. The default settings enable users to enable mixed precision training with minimal code easily.

In addition to the default settings in HMP, users also have the option of overriding these defaults and providing their BF16 and FP32 operator lists by passing them as parameter to HPUPrecisionPlugin.

The below snippet shows an example model using MNIST with a single Habana Gaudi device and making use of HMP by overriding the default parameters. This enables advanced users to provide their own BF16 and FP32 operator list instead of using the HMP defaults.

import pytorch_lightning as pl
from pytorch_lightning.plugins import HPUPrecisionPlugin

# Initialize a trainer with HPU accelerator for HPU strategy for single device,
# with mixed precision using overidden HMP settings
trainer = pl.Trainer(
    accelerator="hpu",
    devices=1,
    # Optional Habana mixed precision params to be set
    # Checkout `examples/pl_hpu/ops_bf16_mnist.txt` for the format
    plugins=[
        HPUPrecisionPlugin(
            precision=16,
            opt_level="O1",
            verbose=False,
            bf16_file_path="ops_bf16_mnist.txt",
            fp32_file_path="ops_fp32_mnist.txt",
        )
    ],
)

# Init our model
model = LitClassifier()
# Init the data
dm = MNISTDataModule(batch_size=batch_size)

# Train the model ⚡
trainer.fit(model, datamodule=dm)

For more details, please refer to PyTorch Mixed Precision Training on Gaudi.


© Copyright Copyright (c) 2018-2022, William Falcon et al... Revision 15b64cf8.

Built with Sphinx using a theme provided by Read the Docs.
Read the Docs v: latest
Versions
latest
stable
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.10
1.5.9
1.5.8
1.5.7
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.10
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.8
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.8
1.0.7
1.0.6
1.0.5
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.10.0
0.9.0
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.0
0.5.3
0.4.9
future-structure
Downloads
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.