Writing a step of your own - xkqg/DeepSharp GitHub Wiki

Writing a step of your own

In short

A step of your own stands exactly where the library's steps stand: it is declared in the chain, written into the pipeline's file under its verb, and read back by a program that has never seen your class once its catalog has been taught the verb. The one rule to remember: a step either adds columns and learns nothing, or learns from the training rows alone and replays what it learned — never both.

You want to… Write
name the verb public static string Name => "feature.weeks";
say what it is for public static string Purpose => "…";
type a key once new ColumnParameter("from", "…", "age", ColumnKinds.Numbers) · new NewColumnParameter("into", "…", "age_in_weeks")
bind the keys to properties new StepParameters<WeeksStep>().With(FromKey, step => step.From)
read it back from its JSON public static WeeksStep ReadFrom(JsonElement element)
add a column, learning nothing IAddsColumns with AddTo(Table table)
learn from the training rows and replay IFittedStep with Fit(Table, IReadOnlyList<Part>) and ApplyTo(Table, FittedStepValues)
say which columns it leaves IDescribesColumns with After(ColumnState before)
put it in a chain .Add(new WeeksStep("Age", "AgeInWeeks"))
teach a catalog the verb catalog.Register<WeeksStep>()
read a file that names it PreparedData.FromJson(json, catalog)

The rest of this page explains each one, with a whole step of each kind.

A step is a record that says four things

Every verb of a pipeline is a small record that names itself, says what it takes, and can be written to a file and read back. The ones this library ships are written that way, and so is the one the indicators package adds.

Member What it says
Name the verb it is written under — lowercase and dotted, like feature.weeks
Purpose one sentence a person reads
Parameters each key, bound to the property that holds it
ReadFrom builds the step back from the JSON it was written as

The parameters are typed keys: a column that must hold numbers, a name for a new column, a number, a share. One list of them writes the step, reads it, checks it, and draws its form in the notebook, so a key is typed in one place.

Make the step a record. Two declarations are compared step by step, and a step that compares by reference makes a pipeline read back from its file unequal to the one it came from.

It says what kind of step it is, by one capability

Capability What the step does Where it may stand
IAddsColumns does arithmetic on the rows and learns nothing from them; it puts new columns on the table above the split
IFittedStep fits on the training rows alone, and applies what it learned to every row only below the split

A step cannot be both, because the run could not place it.

IDescribesColumns is how it says which columns it leaves behind. A declaration can then follow the columns down from the schema, and refuse a step that reads one that is not there.

A step that adds a column and learns nothing

using System.Text.Json;
using DeepSharp.Pipelines;

var catalog = StepCatalog.BuiltIn().WithWeeks();

var prepared = Pdd.Create()
    .ReadCsv("flocks.csv")
    .Declare(schema => schema.Timestamp("Date").Number("Animals", "Age", "Weight"))
    .Add(new WeeksStep("Age", "AgeInWeeks"))              // learns nothing, so it may stand above the split
    .SplitAtRandom(0.70, 0.15)
    .Target("Weight")
    .Drop("Date", "Age")
    .Build()
    .Run();

// The verb is in the file, and a program that has never met the class reads it back with a catalog taught the verb.
var again = PreparedData.FromJson(prepared.ToJson(), catalog);

public sealed record WeeksStep : IPipelineStep<WeeksStep>, IAddsColumns, IDescribesColumns
{
    private static readonly ColumnParameter FromKey = new("from", "The column that holds days.", "age", ColumnKinds.Numbers);
    private static readonly NewColumnParameter IntoKey = new("into", "What the weeks are called.", "age_in_weeks");

    public WeeksStep(string from, string into)
    {
        From = FromKey.Require(from);
        Into = IntoKey.Require(into)!;
    }

    public string From { get; }

    public string Into { get; }

    public static string Name => "feature.weeks";

    public static string Purpose => "Turns a number of days into a number of weeks.";

    public string Verb => Name;

    public static StepParameters<WeeksStep> Parameters { get; } = new StepParameters<WeeksStep>()
        .With(FromKey, step => step.From)
        .With(IntoKey, step => step.Into);

    public ColumnState After(ColumnState before) => before.With(Into, ColumnKind.Number);

    public void AddTo(Table table) =>
        table.Put(new Column<double>(Into, ColumnKind.Number, table.NumbersOf(From).Select(days => days / 7)));

    public static WeeksStep ReadFrom(JsonElement element) => new(FromKey.Read(element), IntoKey.Read(element)!);
}

public static class WeeksExtensions
{
    extension(StepCatalog catalog)
    {
        public StepCatalog WithWeeks()
        {
            catalog.Register<WeeksStep>();

            return catalog;
        }
    }
}

A catalog is taught a verb, and a file is read with a catalog

  • StepCatalog.BuiltIn() knows the verbs this library ships.
  • catalog.Register<TStep>() adds one. Writing it as an extension — WithWeeks() here, WithIndicators() in the indicators package — keeps registration to one sentence.
  • A file that names a verb its catalog was never taught is refused by name, with the nearest verb it does know.
  • An application with a host registers the verb once, for every catalog it builds, by offering an IStepContribution to its services before it calls AddDeepSharpPipelines().

A step that learns, and replays what it learned

using System.Text.Json;
using DeepSharp.Pipelines;

var prepared = Pdd.Create()
    .ReadCsv("flocks.csv")
    .Declare(schema => schema.Timestamp("Date").Number("Animals", "Age", "Weight"))
    .SplitAtRandom(0.70, 0.15)
    .Target("Weight")
    .Drop("Date")
    .Add(new CentredStep("Age", "Age_centred"))             // below the split: it learns from the training rows
    .Build()
    .Run();

Console.WriteLine(prepared.ToJson().Contains("\"mean\"", StringComparison.Ordinal));   // what it learned travels with the file

public sealed record CentredStep : IPipelineStep<CentredStep>, IFittedStep, IDescribesColumns
{
    private static readonly ColumnParameter FromKey = new("from", "The column to centre.", "age", ColumnKinds.Numbers);
    private static readonly NewColumnParameter IntoKey = new("into", "What the centred column is called.", "age_centred");

    public CentredStep(string from, string into)
    {
        From = FromKey.Require(from);
        Into = IntoKey.Require(into)!;
    }

    public string From { get; }

    public string Into { get; }

    public static string Name => "feature.centred";

    public static string Purpose => "Subtracts the mean of the training rows from a column.";

    public string Verb => Name;

    public static StepParameters<CentredStep> Parameters { get; } = new StepParameters<CentredStep>()
        .With(FromKey, step => step.From)
        .With(IntoKey, step => step.Into);

    public ColumnState After(ColumnState before) => before.With(Into, ColumnKind.Number);

    public FittedStepValues Fit(Table table, IReadOnlyList<Part> parts)
    {
        var values = table.NumbersOf(From);
        var mean = Enumerable.Range(0, table.RowCount)
            .Where(row => parts[row] == Part.Train && values[row] is not null)
            .Average(row => values[row]!.Value);
        var learned = new FittedStepValues();

        learned.Learned("mean", mean);

        return learned;
    }

    public void ApplyTo(Table table, FittedStepValues fitted)
    {
        var mean = fitted.Number("mean");

        table.Put(new Column<double>(Into, ColumnKind.Number, table.NumbersOf(From).Select(value => value - mean)));
    }

    public static CentredStep ReadFrom(JsonElement element) => new(FromKey.Read(element), IntoKey.Read(element)!);
}

Fit, then apply

  • Fit is given the table and which part each row is in, and learns from the training rows alone. What it returns is written into the fitted half of the file under the step's key.
  • ApplyTo is given every row and what was learned, and learns nothing more. So validation, test and a row that arrives in production next year are all treated with the same number.
  • What ApplyTo reads is written with Learned — a number, a word, a list of words or a run of numbers — or Decided, for a choice. What a person should see and no replay reads — how many gaps there were, or the training rows themselves, counted, as watch.reference keeps them — is written with Saw: a count, a list of words or a run of numbers. A test of your own can hold a replay to never reading it, as the steps that ship are held.

A step that some learners do without can say so as well, by Pipeline's IMeetsANeed.

Take care with a step that learns from the answer

ApplyTo is handed every row and does not know which of them were the training rows, so a step cannot treat a training row differently from any other. For a column of numbers about the rows themselves that is exactly right.

For a statistic of the answer — a farm's mean weight, a customer's average spend, the history of a group — it is a leak that no test of the step will show. The training row's own answer is inside the group mean it is handed, and a model learns that the feature knows the answer.

What the leak costs, measured

In a simulation with a farm effect of one and a noise of two per flock, three flocks per farm:

The farm's mean, worked out… R² on the rows it was fitted on R² on rows it had not seen
with each row's own answer inside it 0.53 −0.23
with each training row left out of its own farm 0.06 0.05

The second row is the honest size of what the farm tells.

What to do instead

Work such a history out with the row's own answer left out, and for the training rows only, before the data reaches the pipeline. Hand the pipeline the column it made; the pipeline then treats it as the number it is.

Next: Pipeline has every verb the library ships · Saving and replaying says what a file keeps of a fit

⚠️ **GitHub.com Fallback** ⚠️