Development build. This page describes main, not a released package. The latest published Lodestar.Metrics is 0.3.0 — read its documentation.

Home › Metrics › Classification metrics

The binary Brier score — sklearn.metrics.brier_score_loss.

public static double Score(ReadOnlySpan<int> yTrue, ReadOnlySpan<double> yProba, int posLabel = 1, bool scaleByHalf = true, ReadOnlySpan<double> sampleWeight = default)

Parameters — yTrue is the true labels, one per sample. yProba is the probability of posLabel for each sample, in [0, 1]. posLabel is the label that probability is about, 1 by default. scaleByHalf halves the two-class sum, which is what scale_by_half='auto' resolves to for a one-dimensional probability; false doubles the number. sampleWeight is one weight per sample, or empty.

Returns — double in [0, 1] when scaleByHalf holds, [0, 2] when it does not. 0 is a perfect, perfectly confident prediction.

Exceptions — ArgumentException when the lengths disagree, the input is empty, or a probability falls outside [0, 1] — "y_prob contains values greater than 1: 1.5" above, and "y_prob contains values less than 0: -0.1" below, which is this reference's wording where LogLoss.Score's says lower. yTrue holding more than two distinct labels is refused with the reference's "The type of the target inferred from y_true is multiclass but should be binary according to the shape of y_prob." rather than counting the third as negative — BrierScore.MultiClass scores it. A sampleWeight holding NaN or an infinity is refused with "Input sample_weight contains NaN." or its infinity counterpart, and one that is zero throughout with "Sample weights must contain at least one non-zero number." — both ArgumentException naming sampleWeight, as scikit-learn's _check_sample_weight refuses them. Weights that merely sum to zero are refused too, with numpy's "Weights sum to zero, can't be normalized."

Example — the four samples the log-loss page scores, read the other way.

using Lodestar.Metrics;

int[] truth = [0, 1, 1, 0];
double[] confidence = [0.1, 0.9, 0.8, 0.3];

double brier = BrierScore.Score(truth, confidence);  // => 0.0374…

The same input scored about the other class is a different question and a different number:

using Lodestar.Metrics;

int[] truth = [0, 1, 1, 0];
double[] confidence = [0.1, 0.9, 0.8, 0.3];

double aboutZero = BrierScore.Score(truth, confidence, posLabel: 0);  // => 0.6875

Remarks — scikit-learn infers pos_label as the greater of the two labels present, and refuses to guess at all for non-numeric labels; here it is a parameter with a default, as RocAuc.Score's already is. -1/1 and 1/2 labels therefore need no special handling on either side, and both score 0.0375 on the example above.

Applies to — net10.0, netstandard2.0.

See also — BrierScore.MultiClass, LogLoss.Score, the Python equivalence table.