Shortcuts

DeepSpeedPrecisionPlugin

class pytorch_lightning.plugins.precision.DeepSpeedPrecisionPlugin(precision, amp_type, amp_level=None)[source]

Bases: pytorch_lightning.plugins.precision.precision_plugin.PrecisionPlugin

Precision plugin for DeepSpeed integration.

Parameters
  • precision (Union[str, int]) – Double precision (64), full precision (32), half precision (16) or bfloat16 precision (bf16).

  • amp_type (str) – The mixed precision backend to use (“native” or “apex”).

  • amp_level (Optional[str]) – The optimization level to use (O1, O2, etc…). By default it will be set to “O2” if amp_type is set to “apex”.

Raises
  • MisconfigurationException – If using bfloat16 precision and deepspeed<v0.6.

  • ValueError – If unsupported precision is provided.

backward(model, closure_loss, optimizer, optimizer_idx, *args, **kwargs)[source]

Performs back-propagation using DeepSpeed’s engine.

Parameters
  • model (LightningModule) – the model to be optimized

  • closure_loss (Tensor) – the loss tensor

  • optimizer (Optional[Optimizer]) – ignored for DeepSpeed

  • optimizer_idx (Optional[int]) – ignored for DeepSpeed

  • *args – additional positional arguments for the deepspeed.DeepSpeedEngine.backward() call

  • **kwargs – additional keyword arguments for the deepspeed.DeepSpeedEngine.backward() call

Return type

None

clip_gradients(optimizer, clip_val=0.0, gradient_clip_algorithm=GradClipAlgorithmType.NORM)[source]

DeepSpeed handles gradient clipping internally.

Return type

None

optimizer_step(model, optimizer, optimizer_idx, closure, **kwargs)[source]

Hook to run the optimizer step.

Return type

Any

Read the Docs v: latest
Versions
latest
stable
1.7.1
1.7.0
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.10
1.5.9
1.5.8
1.5.7
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.10
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.8
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.8
1.0.7
1.0.6
1.0.5
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.10.0
0.9.0
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.6
0.7.5
0.7.4
0.7.3
0.7.2
0.7.1
0.7.0
0.6.0
0.5.3
0.4.9
future-structure
Downloads
html
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.