Development build. This page describes main, not a released package. The latest published Lodestar.Metrics is 0.4.0 — read its documentation.

Home › Metrics › Classification metrics

Area under the ROC curve for two classes.

public static double Score(ReadOnlySpan<int> yTrue, ReadOnlySpan<double> yScore, int posLabel = 1, ReadOnlySpan<double> sampleWeight = default)

Parameters — yTrue holds the true labels, and exactly two distinct values must occur. yScore holds one score per sample: the higher, the more the model believes posLabel. posLabel is the label counted as positive, 1 by default, which is what scikit-learn infers for 0/1 labels. sampleWeight weights the samples.

Returns — double in [0, 1]: 1 when every positive outranks every negative, 0.5 for a random ranking, and below 0.5 when the ranking is inverted.

Exceptions — ArgumentException, first, when either span is empty or a score is not finite, in check_array's words — "Found array with 0 sample(s)…", "Input contains NaN." — before any label is read (#1585); then when yTrue holds more than two labels, which roc_auc_score refuses without multi_class; when it holds two and neither is posLabel — the reference would score the greater one, and posLabel always names one here (#1277); or, over two labels, when the spans disagree in length or a weight is not finite. One class alone scores NaN before a length or a weight is looked at, and a class weighted zero throughout scores NaN too, as roc_auc_score 1.9.1 answers after its warning (#1584). Over two labels a negative weight that turns the false-positive rate back is refused with auc's "x is neither increasing nor decreasing", negative samples whose weights total zero or less give NaN, as roc_curve makes its rates, and so do positive samples whose weights total zero or less unless the false-positive rate turns back (#1601).

Example — four samples and the model's confidence in each.

using Lodestar.Metrics;

int[] yTrue = [0, 0, 1, 1];
double[] yScore = [0.1, 0.4, 0.35, 0.8];

double auc = RocAuc.Score(yTrue, yScore);   // => 0.75

Remarks — everything else on this page scores a decision; this scores a ranking. That is the reason to pick it: it needs no threshold, so it measures the model rather than the cut-off someone chose afterwards, and a model whose scores are well ordered but badly calibrated still scores well. The number has a direct reading — take one positive and one negative at random, and this is the probability the positive got the higher score.

0.5 is the floor that matters, not 0. A score below 0.5 does not mean a bad model so much as a sign flip: 1 - auc is what you would get by ranking the other way.

Two traps. This is insensitive to class imbalance in a way that can flatter a model: with 1% positives, a model can score 0.95 here and still have a precision near zero at every useful threshold, because the negatives it ranks above the positives are so numerous. If you are choosing an operating point rather than comparing models, look at precision and recall at the threshold you will actually use. And posLabel is explicit here where scikit-learn infers it; the default of 1 is what it infers for 0/1 labels, so labels like [-1, 1] or [1, 2] need it said.

Applies to — net10.0, netstandard2.0.

See also — RocAuc.MultiClass, Precision.Score, Recall.Score, the Python equivalence table.