Preprocessing robustscaleroptions - CyrilB1531/lodestar GitHub Wiki

Development build. This page describes main, not a released package. The latest published Lodestar.Preprocessing is 0.2.0 — read its documentation.

Home › Preprocessing › Feature scaling

RobustScalerOptions

Which steps RobustScaler applies, and between which percentiles it scales.

public sealed record RobustScalerOptions

Properties — WithCentring subtracts each feature's median and WithScaling divides by its interpercentile range, both on by default. LowerPercentile and UpperPercentile are that range, defaulting to the quartiles. UnitVariance divides that range again, by the normal quantiles of the two percentiles, so a normal column comes out with a standard deviation of 1 — the reference's with_centering, with_scaling, quantile_range and unit_variance, same defaults.

Example — the deciles instead of the quartiles, centring turned off, and unit variance.

using Lodestar.Preprocessing;

double[] samples = [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0];

RobustScaler deciles = RobustScaler.Fit(
    samples, 1, new RobustScalerOptions { LowerPercentile = 10.0, UpperPercentile = 90.0 });

double wider = deciles.Scale![0];  // => 8

RobustScaler uncentred = RobustScaler.Fit(
    samples, 1, new RobustScalerOptions { WithCentring = false });

bool noMedian = uncentred.Centre is null;  // => True

// 1.3489795 is the normal quantile gap at the quartiles, so the quartile range of 5 becomes 3.7065.
RobustScaler unit = RobustScaler.Fit(
    samples, 1, new RobustScalerOptions { UnitVariance = true });

double standardised = unit.Scale![0];  // => 3.706505546264005

Remarks — each switch decides exactly one statistic, which is worth stating because StandardScalerOptions's pair does not: there, turning centring off still fits a mean. Here, WithCentring = false leaves Centre null and nothing else moves.

A wider percentile range makes the scale larger and the scaled values smaller; the reference offers it for the same reason it offers the quartiles, and refuses anything outside 0 ≤ lower ≤ upper ≤ 100.

The last digit is scipy's, 3.706505546264005: the normal quantile is scipy.special.ndtri ported statement for statement (#1684), where an inversion of the tail landed two units in the last place away.

UnitVariance divides the range after the near-constant floor, not before, which is the reference's order and is visible only on a constant feature: it lands on 1/1.3489795, not on 1. The quantile is Lodestar.Stats' published one — the edge decision 0003 took rather than carry a second copy. A percentile of 0 or 100 is refused with it, having no finite quantile, where the reference divides by an infinity and reports a scale of zero.

Applies to — net10.0, netstandard2.0.

See also — RobustScaler, RobustScaler.Fit.

Members

Member What it does
RobustScalerOptions.GetHashCode A hash consistent with the equality, every NaN alike.