pytorch_lightning.core.grads module¶
Module to describe gradients
-
class
pytorch_lightning.core.grads.GradInformation(*args, **kwargs)[source]¶ Bases:
torch.nn.Module-
grad_norm(norm_type)[source]¶ Compute each parameter’s gradient’s norm and their overall norm.
The overall norm is computed over all gradients together, as if they were concatenated into a single vector.
- Parameters
norm_type¶ (
Union[float,int,str]) – The type of the used p-norm, cast to float if necessary. Can be'inf'for infinity norm.- Returns
- The dictionary of p-norms of each parameter’s gradient and
a special entry for the total p-norm of the gradients viewed as a single vector.
- Return type
norms
-
