pytorch_lightning.metrics package¶
-
class
pytorch_lightning.metrics.MAE(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.MetricComputes the mean absolute loss or L1-loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = MAE() >>> metric(pred, target) tensor(0.2500)
-
class
pytorch_lightning.metrics.MSE(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.MetricComputes the mean squared loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = MSE() >>> metric(pred, target) tensor(0.2500)
-
class
pytorch_lightning.metrics.PSNR(data_range=None, base=10, reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.MetricComputes the peak signal-to-noise ratio
Example
>>> pred = torch.tensor([[0.0, 1.0], [2.0, 3.0]]) >>> target = torch.tensor([[3.0, 2.0], [1.0, 0.0]]) >>> metric = PSNR() >>> metric(pred, target) tensor(2.5527)
- Parameters
-
class
pytorch_lightning.metrics.RMSE(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.MetricComputes the root mean squared loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = RMSE() >>> metric(pred, target) tensor(0.5000)
-
class
pytorch_lightning.metrics.RMSLE(reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.MetricComputes the root mean squared log loss.
Example
>>> pred = torch.tensor([0., 1, 2, 3]) >>> target = torch.tensor([0., 1, 2, 2]) >>> metric = RMSLE() >>> metric(pred, target) tensor(0.0207)
-
class
pytorch_lightning.metrics.SSIM(kernel_size=(11, 11), sigma=(1.5, 1.5), reduction='elementwise_mean', data_range=None, k1=0.01, k2=0.03)[source]¶ Bases:
pytorch_lightning.metrics.metric.MetricComputes Structual Similarity Index Measure
Example
>>> pred = torch.rand([16, 1, 16, 16]) >>> target = pred * 0.75 >>> metric = SSIM() >>> metric(pred, target) tensor(0.9219)
- Parameters
kernel_size¶ (
Sequence[int]) – Size of the gaussian kernel (default: (11, 11))sigma¶ (
Sequence[float]) – Standard deviation of the gaussian kernel (default: (1.5, 1.5))reduction¶ (
str) – a method to reduce metric score over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass away - sum: add elementsdata_range¶ (
Optional[float]) – Range of the image. IfNone, it is determined from the image (max - min)
-
class
pytorch_lightning.metrics.AUC(reduce_group=torch.distributed.group.WORLD, reduce_op=torch.distributed.ReduceOp.SUM)[source]¶ Bases:
pytorch_lightning.metrics.sklearns.SklearnMetricCalculates the Area Under the Curve using the trapoezoidal rule
Warning
Every metric call will cause a GPU synchronization, which may slow down your code
Example
>>> y_pred = torch.tensor([0, 1, 2, 3]) >>> y_true = torch.tensor([0, 1, 2, 2]) >>> metric = AUC() >>> metric(y_pred, y_true) tensor([4.])
- Parameters
-
class
pytorch_lightning.metrics.AUROC(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the area under curve (AUC) of the receiver operator characteristic (ROC)
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = AUROC() >>> metric(pred, target) tensor(0.3333)
- Parameters
-
class
pytorch_lightning.metrics.Accuracy(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the accuracy classification score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = Accuracy() >>> metric(pred, target) tensor(0.7500)
- Parameters
reduction¶ (
str) – a method to reduce metric score over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional[Any]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional[Any]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.AveragePrecision(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the average precision score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = AveragePrecision() >>> metric(pred, target) tensor(0.3333)
- Parameters
-
class
pytorch_lightning.metrics.ConfusionMatrix(normalize=False, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the confusion matrix C where each entry C_{i,j} is the number of observations in group i that were predicted in group j.
Example
>>> pred = torch.tensor([0, 1, 2, 2]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = ConfusionMatrix() >>> metric(pred, target) tensor([[1., 0., 0.], [0., 1., 0.], [0., 0., 2.]])
- Parameters
-
class
pytorch_lightning.metrics.DiceCoefficient(include_background=False, nan_score=0.0, no_fg_score=0.0, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the dice coefficient
Example
>>> pred = torch.tensor([[0.85, 0.05, 0.05, 0.05], ... [0.05, 0.85, 0.05, 0.05], ... [0.05, 0.05, 0.85, 0.05], ... [0.05, 0.05, 0.05, 0.85]]) >>> target = torch.tensor([0, 1, 3, 2]) >>> metric = DiceCoefficient() >>> metric(pred, target) tensor(0.3333)
- Parameters
include_background¶ (
bool) – whether to also compute dice for the backgroundnan_score¶ (
float) – score to return, if a NaN occurs during computation (denom zero)no_fg_score¶ (
float) – score to return, if no foreground pixel was found in targetreduction¶ (
str) – a method to reduce metric score over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional[Any]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional[Any]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.F1(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the F1 score, which is the harmonic mean of the precision and recall. It ranges between 1 and 0, where 1 is perfect and the worst value is 0.
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = F1() >>> metric(pred, target) tensor(0.6667)
- Parameters
reduction¶ (
str) – a method to reduce metric score over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional[Any]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional[Any]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.FBeta(beta, num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetric- Computes the FBeta Score, which is the weighted harmonic mean of precision and recall.
It ranges between 1 and 0, where 1 is perfect and the worst value is 0.
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = FBeta(0.25) >>> metric(pred, target) tensor(0.7361)
- Parameters
beta¶ (
float) – determines the weight of recall in the combined score.reduction¶ (
str) – a method to reduce metric score over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional[Any]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional[Any]) – the operation to perform for DDP reduction
-
class
pytorch_lightning.metrics.MulticlassPrecisionRecall(num_classes=None, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetricComputes the multiclass PR Curve
Example
>>> pred = torch.tensor([[0.85, 0.05, 0.05, 0.05], ... [0.05, 0.85, 0.05, 0.05], ... [0.05, 0.05, 0.85, 0.05], ... [0.05, 0.05, 0.05, 0.85]]) >>> target = torch.tensor([0, 1, 3, 2]) >>> metric = MulticlassPrecisionRecall() >>> metric(pred, target) ((tensor([1., 1.]), tensor([1., 0.]), tensor([0.8500])), (tensor([1., 1.]), tensor([1., 0.]), tensor([0.8500])), (tensor([0.2500, 0.0000, 1.0000]), tensor([1., 0., 0.]), tensor([0.0500, 0.8500])), (tensor([0.2500, 0.0000, 1.0000]), tensor([1., 0., 0.]), tensor([0.0500, 0.8500])))
- Parameters
-
class
pytorch_lightning.metrics.MulticlassROC(num_classes=None, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetricComputes the multiclass ROC
Example
>>> pred = torch.tensor([[0.85, 0.05, 0.05, 0.05], ... [0.05, 0.85, 0.05, 0.05], ... [0.05, 0.05, 0.85, 0.05], ... [0.05, 0.05, 0.05, 0.85]]) >>> target = torch.tensor([0, 1, 3, 2]) >>> metric = MulticlassROC() >>> classes_roc = metric(pred, target) >>> metric(pred, target) ((tensor([0., 0., 1.]), tensor([0., 1., 1.]), tensor([1.8500, 0.8500, 0.0500])), (tensor([0., 0., 1.]), tensor([0., 1., 1.]), tensor([1.8500, 0.8500, 0.0500])), (tensor([0.0000, 0.3333, 1.0000]), tensor([0., 0., 1.]), tensor([1.8500, 0.8500, 0.0500])), (tensor([0.0000, 0.3333, 1.0000]), tensor([0., 0., 1.]), tensor([1.8500, 0.8500, 0.0500])))
- Parameters
-
class
pytorch_lightning.metrics.Precision(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the precision score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = Precision(num_classes=4) >>> metric(pred, target) tensor(0.7500)
- Parameters
reduction¶ (
str) – a method to reduce metric score over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional[Any]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional[Any]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.PrecisionRecall(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetricComputes the precision recall curve
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = PrecisionRecall() >>> prec, recall, thr = metric(pred, target) >>> prec tensor([0.3333, 0.0000, 0.0000, 1.0000]) >>> recall tensor([1., 0., 0., 0.]) >>> thr tensor([1., 2., 3.])
- Parameters
-
class
pytorch_lightning.metrics.PrecisionRecallCurve(pos_label=1, reduce_group=torch.distributed.group.WORLD, reduce_op=torch.distributed.ReduceOp.SUM)[source]¶ Bases:
pytorch_lightning.metrics.sklearns.SklearnMetricCompute precision-recall pairs for different probability thresholds
Note
This implementation is restricted to the binary classification task.
The precision is the ratio
tp / (tp + fp)wheretpis the number of true positives andfpthe number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The recall is the ratiotp / (tp + fn)wheretpis the number of true positives andfnthe number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples. The last precision and recall values are 1. and 0. respectively and do not have a corresponding threshold. This ensures that the graph starts on the x axis.- Parameters
pos_label¶ (
Union[str,int]) – The class to report ifaverage='binary'.reduce_group¶ (
Any) – the process group for DDP reduces (only needed for DDP training). Defaults to all processes (world)reduce_op¶ (
Any) – the operation to perform during reduction within DDP (only needed for DDP training). Defaults to sum.
-
forward(probas_pred, y_true, sample_weight=None)[source]¶ - Parameters
- Returns
- Precision values such that element i is the precision of
predictions with score >= thresholds[i] and the last element is 1.
- recall:
Decreasing recall values such that element i is the recall of predictions with score >= thresholds[i] and the last element is 0.
- thresholds:
Increasing thresholds on the decision function used to compute precision and recall.
- Return type
precision
-
class
pytorch_lightning.metrics.ROC(pos_label=1, reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorCollectionMetricComputes the Receiver Operator Characteristic (ROC)
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = ROC() >>> fps, tps, thresholds = metric(pred, target) >>> fps tensor([0.0000, 0.3333, 0.6667, 0.6667, 1.0000]) >>> tps tensor([0., 0., 0., 1., 1.]) >>> thresholds tensor([4., 3., 2., 1., 0.])
- Parameters
-
class
pytorch_lightning.metrics.Recall(num_classes=None, reduction='elementwise_mean', reduce_group=None, reduce_op=None)[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the recall score
Example
>>> pred = torch.tensor([0, 1, 2, 3]) >>> target = torch.tensor([0, 1, 2, 2]) >>> metric = Recall() >>> metric(pred, target) tensor(0.6250)
- Parameters
reduction¶ (
str) – a method to reduce metric score over labels (default: takes the mean) Available reduction methods: - elementwise_mean: takes the mean - none: pass array - sum: add elementsreduce_group¶ (
Optional[Any]) – the process group to reduce metric results from DDPreduce_op¶ (
Optional[Any]) – the operation to perform for ddp reduction
-
class
pytorch_lightning.metrics.IoU(remove_bg=False, reduction='elementwise_mean')[source]¶ Bases:
pytorch_lightning.metrics.metric.TensorMetricComputes the intersection over union.
Example
>>> pred = torch.tensor([[0, 0, 0, 0, 0, 0, 0, 0], ... [0, 0, 1, 1, 1, 0, 0, 0], ... [0, 0, 0, 0, 0, 0, 0, 0]]) >>> target = torch.tensor([[0, 0, 0, 0, 0, 0, 0, 0], ... [0, 0, 0, 1, 1, 1, 0, 0], ... [0, 0, 0, 0, 0, 0, 0, 0]]) >>> metric = IoU() >>> metric(pred, target) tensor(0.7045)
- Parameters
remove_bg¶ (
bool) – Flag to state whether a background class has been included within input parameters. If true, will remove background class. If false, return IoU over all classes. Assumes that background is ‘0’ class in input tensora method to reduce metric score over labels (default: takes the mean) Available reduction methods:
elementwise_mean: takes the mean
none: pass array
sum: add elements
-
class
pytorch_lightning.metrics.SklearnMetric(metric_name, reduce_group=torch.distributed.group.WORLD, reduce_op=torch.distributed.ReduceOp.SUM, **kwargs)[source]¶ Bases:
pytorch_lightning.metrics.metric.NumpyMetricBridge between PyTorch Lightning and scikit-learn metrics
Warning
Every metric call will cause a GPU synchronization, which may slow down your code
Note
The order of targets and predictions may be different from the order typically used in PyTorch
- Parameters
metric_name¶ (
str) – the metric name to import and compute from scikit-learn.metricsreduce_group¶ (
Any) – the process group for DDP reduces (only needed for DDP training). Defaults to all processes (world)reduce_op¶ (
Any) – the operation to perform during reduction within DDP (only needed for DDP training). Defaults to sum.**kwargs¶ – additonal keyword arguments (will be forwarded to metric call)
-
class
pytorch_lightning.metrics.BLEUScore(n_gram=4, smooth=False)[source]¶ Bases:
pytorch_lightning.metrics.metric.MetricCalculate BLEU score of machine translated text with one or more references.
Example
>>> translate_corpus = ['the cat is on the mat'.split()] >>> reference_corpus = [['there is a cat on the mat'.split(), 'a cat is on the mat'.split()]] >>> metric = BLEUScore() >>> metric(translate_corpus, reference_corpus) tensor(0.7598)
- Parameters
