Preprocessing splitters - CyrilB1531/lodestar GitHub Wiki

Development build. This page describes main, not a released package. The latest published Lodestar.Preprocessing is 0.1.0 — read its documentation.

HomePreprocessingSplitting

Splitters

Cross-validation and train/test splits over row indices.

public static class Splitters

Example — three stratified folds over twelve rows in three classes.

using Lodestar.Preprocessing;

// Six rows of class 0, three of class 1, three of class 2.
int[] labels = [0, 0, 0, 0, 0, 0, 1, 1, 1, 2, 2, 2];

IReadOnlyList<FoldSplit> folds = Splitters.StratifiedKFold(labels, foldCount: 3);

string held = string.Join(",", folds[0].TestIndices);  // => 0,1,6,9
string fit = string.Join(",", folds[0].TrainIndices);  // => 2,3,4,5,7,8,10,11

Remarks — every fold above holds two rows of class 0 and one of each other class, which is what stratifying buys: Splitters.KFold on the same twelve rows would hand fold 0 four rows of class 0 and nothing else.

The unshuffled splitters are scikit-learn's, fold for fold and index for index, replayed from tests/oracles/preprocessing_splitters.json. The shuffled ones take the permutation as an argument rather than a seed — the splitting index has why, and which shuffled splits a permutation reproduces.

Applies to — net10.0, netstandard2.0.

See alsoFoldSplit, TrainTestSplit, the splitting index, the Python equivalence table.

Members

Member What it does
Splitters.KFold Cuts the rows into contiguous folds.
Splitters.StratifiedKFold Cuts folds that keep each class's share.
Splitters.TrainTest Holds out the last rows as a test set.
⚠️ **GitHub.com Fallback** ⚠️