Preprocessing 0.1.0 scaling - CyrilB1531/lodestar GitHub Wiki

Lodestar.Preprocessing 0.1.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

Feature scaling — Lodestar.Preprocessing

One type, StandardScaler: it centres each feature on its mean and scales it to unit variance, at sklearn.preprocessing.StandardScaler parity.

Spans in, arrays out. A sample matrix is row-major — FeatureCount values per row — which is the shape Lodestar.Metrics already uses, so a matrix does not have to be reshaped to cross between the two packages. There is no pipeline object to build, no data view to construct, and nothing to adopt beyond the call.

Why this exists when ML.NET has normalizers

ML.NET has every brick this package will grow: NormalizeMeanVariance, NormalizeMinMax, OneHotEncoding, ReplaceMissingValues, TrainTestSplit, CrossValidationSplit. Read on Microsoft.ML 5.0.0's exported surface, each one of them is reached through an IDataView or through a catalog naming columnsNormalizeMeanVariance(TransformsCatalog, string inputColumn, string outputColumn, …) never sees a value, and TrainTestSplit(IDataView, double, …) takes and returns data views. The capability is not missing; the array-in, array-out entry point is.

That is the same shape as TfidfVectorizer against FeaturizeText, and the same reason: a caller who holds a double[] and wants a double[] back should not have to adopt a framework to get one.

Types

Type What it is
StandardScaler Centres and scales each feature, and reports the statistics it fitted.
StandardScalerOptions Which of the two steps to apply.

See also