pytorch_lightning.metrics package¶
-
class
pytorch_lightning.metrics.
MAE
(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.Metric
Computes the root mean absolute loss or L1-loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = MAE() >>> metric(pred, target) tensor(0.2500)
- Parameters
reduction¶ (
str
) – a method for reducing mse over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elements
-
class
pytorch_lightning.metrics.
MSE
(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.Metric
Computes the mean squared loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = MSE() >>> metric(pred, target) tensor(0.2500)
- Parameters
reduction¶ (
str
) – a method for reducing mse over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elements
-
class
pytorch_lightning.metrics.
PSNR
(data_range=None, base=10, reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.Metric
Computes the peak signal-to-noise ratio
Example
>>> pred = torch.tensor([[0.0, 1.0], [2.0, 3.0]]) >>> target = torch.tensor([[3.0, 2.0], [1.0, 0.0]]) >>> metric = PSNR() >>> metric(pred, target) tensor(2.5527)
- Parameters
-
class
pytorch_lightning.metrics.
RMSE
(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.Metric
Computes the root mean squared loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = RMSE() >>> metric(pred, target) tensor(0.5000)
- Parameters
reduction¶ (
str
) – a method for reducing mse over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elements
-
class
pytorch_lightning.metrics.
RMSLE
(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.Metric
Computes the root mean squared log loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = RMSLE() >>> metric(pred, target) tensor(0.0207)
- Parameters
reduction¶ (
str
) – a method for reducing mse over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elements
-
class
pytorch_lightning.metrics.
AUC
(reduce_group=torch.distributed.group.WORLD, reduce_op=torch.distributed.ReduceOp.SUM)[source]¶ Bases:
pytorch_lightning.metrics.sklearns.SklearnMetric
Calculates the Area Under the Curve using the trapoezoidal rule
Warning
Every metric call will cause a GPU synchronization, which may slow down your code
Example
>>> y_pred = torch.tensor([0, 1, 2, 3]) >>> y_true = torch.tensor([0, 1, 2, 2]) >>> metric = AUC() >>> metric(y_pred, y_true) tensor([4.])
- Parameters
-
class
pytorch_lightning.metrics.
AUROC
(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the area under curve (AUC) of the receiver operator characteristic (ROC)
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = AUROC() >>> metric(pred, target) tensor(0.3333)
- Parameters
-
class
pytorch_lightning.metrics.
Accuracy
(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the accuracy classification score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = Accuracy() >>> metric(pred, target) tensor(0.7500)
- Parameters
reduction¶ (
str
) – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.
AveragePrecision
(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the average precision score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = AveragePrecision() >>> metric(pred, target) tensor(0.3333)
- Parameters
-
class
pytorch_lightning.metrics.
ConfusionMatrix
(normalize=False, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the confusion matrix C where each entry C_{i,j} is the number of observations in group i that were predicted in group j.
Example
>>> pred = torch.tensor([0, 1, 2, 2]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = ConfusionMatrix() >>> metric(pred, target) tensor([[1., 0., 0.], [0., 1., 0.], [0., 0., 2.]])
- Parameters
-
class
pytorch_lightning.metrics.
DiceCoefficient
(include_background=False, nan_score=0.0, no_fg_score=0.0, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the dice coefficient
Example
>>> pred = torch.tensor([[0.85, 0.05, 0.05, 0.05], ... [0.05, 0.85, 0.05, 0.05], ... [0.05, 0.05, 0.85, 0.05], ... [0.05, 0.05, 0.05, 0.85]]) >>> target = torch.tensor([0, 1, 3, 2]) >>> metric = DiceCoefficient() >>> metric(pred, target) tensor(0.3333)
- Parameters
include_background¶ (
bool
) – whether to also compute dice for the backgroundnan_score¶ (
float
) – score to return, if a NaN occurs during computation (denom zero)no_fg_score¶ (
float
) – score to return, if no foreground pixel was found in targetreduction¶ (
str
) – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.
F1
(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the F1 score, which is the harmonic mean of the precision and recall. It ranges between 1 and 0, where 1 is perfect and the worst value is 0.
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = F1() >>> metric(pred, target) tensor(0.6667)
- Parameters
reduction¶ (
str
) – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.
FBeta
(beta, num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
- Computes the FBeta Score, which is the weighted harmonic mean of precision and recall.
It ranges between 1 and 0, where 1 is perfect and the worst value is 0.
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = FBeta(0.25) >>> metric(pred, target) tensor(0.7361)
- Parameters
beta¶ (
float
) – determines the weight of recall in the combined score.reduction¶ (
str
) – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for DDP reduction
-
class
pytorch_lightning.metrics.
MulticlassPrecisionRecall
(num_classes=None, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetric
Computes the multiclass PR Curve
Example
>>> pred = torch.tensor([[0.85, 0.05, 0.05, 0.05], ... [0.05, 0.85, 0.05, 0.05], ... [0.05, 0.05, 0.85, 0.05], ... [0.05, 0.05, 0.05, 0.85]]) >>> target = torch.tensor([0, 1, 3, 2]) >>> metric = MulticlassPrecisionRecall() >>> metric(pred, target) ((tensor([1., 1.]), tensor([1., 0.]), tensor([0.8500])), (tensor([1., 1.]), tensor([1., 0.]), tensor([0.8500])), (tensor([0.2500, 0.0000, 1.0000]), tensor([1., 0., 0.]), tensor([0.0500, 0.8500])), (tensor([0.2500, 0.0000, 1.0000]), tensor([1., 0., 0.]), tensor([0.0500, 0.8500])))
- Parameters
reduction¶ – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elements
reduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.
MulticlassROC
(num_classes=None, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetric
Computes the multiclass ROC
Example
>>> pred = torch.tensor([[0.85, 0.05, 0.05, 0.05], ... [0.05, 0.85, 0.05, 0.05], ... [0.05, 0.05, 0.85, 0.05], ... [0.05, 0.05, 0.05, 0.85]]) >>> target = torch.tensor([0, 1, 3, 2]) >>> metric = MulticlassROC() >>> classes_roc = metric(pred, target) >>> metric(pred, target) ((tensor([0., 0., 1.]), tensor([0., 1., 1.]), tensor([1.8500, 0.8500, 0.0500])), (tensor([0., 0., 1.]), tensor([0., 1., 1.]), tensor([1.8500, 0.8500, 0.0500])), (tensor([0.0000, 0.3333, 1.0000]), tensor([0., 0., 1.]), tensor([1.8500, 0.8500, 0.0500])), (tensor([0.0000, 0.3333, 1.0000]), tensor([0., 0., 1.]), tensor([1.8500, 0.8500, 0.0500])))
- Parameters
reduction¶ – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elements
reduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.
Precision
(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the precision score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = Precision(num_classes=4) >>> metric(pred, target) tensor(0.7500)
- Parameters
reduction¶ (
str
) – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.
PrecisionRecall
(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetric
Computes the precision recall curve
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = PrecisionRecall() >>> prec, recall, thr = metric(pred, target) >>> prec tensor([0.3333, 0.0000, 0.0000, 1.0000]) >>> recall tensor([1., 0., 0., 0.]) >>> thr tensor([1., 2., 3.])
- Parameters
-
class
pytorch_lightning.metrics.
PrecisionRecallCurve
(pos_label=1, reduce_group=torch.distributed.group.WORLD, reduce_op=torch.distributed.ReduceOp.SUM)[source]¶ Bases:
pytorch_lightning.metrics.sklearns.SklearnMetric
Compute precision-recall pairs for different probability thresholds
Note
This implementation is restricted to the binary classification task.
The precision is the ratio
tp / (tp + fp)
wheretp
is the number of true positives andfp
the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The recall is the ratiotp / (tp + fn)
wheretp
is the number of true positives andfn
the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples. The last precision and recall values are 1. and 0. respectively and do not have a corresponding threshold. This ensures that the graph starts on the x axis.- Parameters
pos_label¶ (
Union
[str
,int
]) – The class to report ifaverage='binary'
.reduce_group¶ (
Any
) – the process group for DDP reduces (only needed for DDP training). Defaults to all processes (world)reduce_op¶ (
Any
) – the operation to perform during reduction within DDP (only needed for DDP training). Defaults to sum.
-
forward
(probas_pred, y_true, sample_weight=None)[source]¶ - Parameters
- Returns
- Precision values such that element i is the precision of
predictions with score >= thresholds[i] and the last element is 1.
- recall:
Decreasing recall values such that element i is the recall of predictions with score >= thresholds[i] and the last element is 0.
- thresholds:
Increasing thresholds on the decision function used to compute precision and recall.
- Return type
precision
-
class
pytorch_lightning.metrics.
ROC
(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetric
Computes the Receiver Operator Characteristic (ROC)
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = ROC() >>> fps, tps, thresholds = metric(pred, target) >>> fps tensor([0.0000, 0.3333, 0.6667, 0.6667, 1.0000]) >>> tps tensor([0., 0., 0., 1., 1.]) >>> thresholds tensor([4., 3., 2., 1., 0.])
- Parameters
-
class
pytorch_lightning.metrics.
Recall
(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the recall score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = Recall() >>> metric(pred, target) tensor(0.6250)
- Parameters
reduction¶ (
str
) – a method for reducing accuracies over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional
[Any
]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional
[Any
]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.
IoU
(remove_bg=False, reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric
Computes the intersection over union.
Example
>>> pred = torch.tensor([[0, 0, 0, 0, 0, 0, 0, 0], ... [0, 0, 1, 1, 1, 0, 0, 0], ... [0, 0, 0, 0, 0, 0, 0, 0]]) >>> target = torch.tensor([[0, 0, 0, 0, 0, 0, 0, 0], ... [0, 0, 0, 1, 1, 1, 0, 0], ... [0, 0, 0, 0, 0, 0, 0, 0]]) >>> metric = IoU() >>> metric(pred, target) tensor(0.7045)
- Parameters
remove_bg¶ (
bool
) – Flag to state whether a background class has been included within input parameters. If true, will remove background class. If false, return IoU over all classes. Assumes that background is ‘0’ class in input tensora method for reducing IoU over labels (default: takes the mean) Available reduction methods:
elementwise_mean: takes the mean
none: pass array
sum: add elements
-
class
pytorch_lightning.metrics.
SklearnMetric
(metric_name, reduce_group=torch.distributed.group.WORLD, reduce_op=torch.distributed.ReduceOp.SUM, **kwargs)[source]¶ Bases:
pytorch_lightning.metrics.metric.NumpyMetric
Bridge between PyTorch Lightning and scikit-learn metrics
Warning
Every metric call will cause a GPU synchronization, which may slow down your code
Note
The order of targets and predictions may be different from the order typically used in PyTorch
- Parameters
metric_name¶ (
str
) – the metric name to import and compute from scikit-learn.metricsreduce_group¶ (
Any
) – the process group for DDP reduces (only needed for DDP training). Defaults to all processes (world)reduce_op¶ (
Any
) – the operation to perform during reduction within DDP (only needed for DDP training). Defaults to sum.**kwargs¶ – additonal keyword arguments (will be forwarded to metric call)