Development build. This page describes main, not a released package. The latest published Lodestar.Metrics is 0.4.0 — read its documentation.

Home › Metrics › Regression metrics

The fraction of pinball loss explained — sklearn.metrics.d2_pinball_score.

public static double Score(ReadOnlySpan<double> yTrue, ReadOnlySpan<double> yPred, double alpha = 0.5, int outputCount = 1, ReadOnlySpan<double> sampleWeight = default, ReadOnlySpan<double> outputWeights = default, ZeroDivision zeroDivision = ZeroDivision.NaN)

Parameters — yTrue and yPred are the true and predicted values, row-major when there is more than one output. alpha is the quantile being scored, in [0, 1], 0.5 by default. outputCount is how many outputs each row holds. sampleWeight is one weight per row, not per value. outputWeights is a weight per output, multioutput=[…]; omit it for multioutput="uniform_average". zeroDivision decides the answer for fewer than two samples.

Returns — double. 1 for a perfect prediction, 0 for one no better than the best constant, and negative below that. A column whose truth never varies contributes 0.

Exceptions — ArgumentException when a length disagrees with the shape, the input is empty, or it holds a non-finite value. ArgumentOutOfRangeException when outputCount is below one, or alpha is outside [0, 1] — NaN included. UndefinedMetricException when there are fewer than two samples and zeroDivision is ZeroDivision.Throw. outputWeights holding NaN or an infinity is ArgumentException too, with scikit-learn's "Input contains NaN." or its infinity counterpart (#1461), and so is any outputWeights on a single output, with "Custom weights are useful only in multi-output cases." (#1533).

Example — the same four samples read at three quantiles.

using Lodestar.Metrics;

double[] truth = [1.0, 2.0, 3.0, 4.0];
double[] predicted = [1.5, 2.5, 2.0, 4.5];

double median = D2Pinball.Score(truth, predicted);  // => 0.375

using Lodestar.Metrics;

double[] truth = [1.0, 2.0, 3.0, 4.0];
double[] predicted = [1.5, 2.5, 2.0, 4.5];

double upper = D2Pinball.Score(truth, predicted, alpha: 0.9);  // => -0.75…

The upper quantile scores below zero: these predictions are a decent median and a poor 90th percentile, which is the distinction the metric exists to draw.

Remarks — the denominator is the loss of predicting the weighted quantile of the truth at the same alpha, taken as scikit-learn's _weighted_percentile(…, average=True) takes it, negative sample weights included: alpha: 0.3 with weights [1, -1, -0.5, -0.5, 2, 1] gives -4.429203539823007 on both sides (#1546). Equal truth values carrying different negative weights follow numpy's argsort order, which is not stable and depends on the CPU, as MedianAbsoluteError.Score describes.

Applies to — net10.0, netstandard2.0.

See also — D2Pinball.PerOutput, D2AbsoluteError.Score, PinballLoss.Score, the Python equivalence table.