Shortcuts

DeepSpeedPrecisionPlugin

class pytorch_lightning.plugins.precision.DeepSpeedPrecisionPlugin(precision, amp_type=None, amp_level=None)[source]

Bases: pytorch_lightning.plugins.precision.precision_plugin.PrecisionPlugin

Precision plugin for DeepSpeed integration.

Parameters

precision (Literal[‘32’, 32, ‘16’, 16, ‘bf16’]) – Full precision (32), half precision (16) or bfloat16 precision (bf16).

Raises

ValueError – If unsupported precision is provided.

backward(tensor, model, optimizer, optimizer_idx, *args, **kwargs)[source]

Performs back-propagation using DeepSpeed’s engine.

Parameters
  • tensor (Tensor) – the loss tensor

  • model (LightningModule) – the model to be optimized

  • optimizer (Optional[Steppable]) – ignored for DeepSpeed

  • optimizer_idx (Optional[int]) – ignored for DeepSpeed

  • *args – additional positional arguments for the deepspeed.DeepSpeedEngine.backward() call

  • **kwargs – additional keyword arguments for the deepspeed.DeepSpeedEngine.backward() call

Return type

None

clip_gradients(optimizer, clip_val=0.0, gradient_clip_algorithm=GradClipAlgorithmType.NORM)[source]

DeepSpeed handles gradient clipping internally.

Return type

None

optimizer_step(optimizer, model, optimizer_idx, closure, **kwargs)[source]

Hook to run the optimizer step.

Return type

Any

Read the Docs v: stable
Versions
latest
stable
1.8.3post1
1.8.3.post0
1.8.3
1.8.2
1.8.1
1.8.0.post1
1.8.0
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.10
1.5.9
1.5.8
1.5.7
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.10
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.8
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.8
1.0.7
1.0.6
1.0.5
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.10.0
0.9.0
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.0
0.5.3
0.4.9
Downloads
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.