Step 6 — Naming the answer

In short

A pipeline says what a model is asked to predict, in the declaration rather than in the code that trains. The handover then knows which column is the answer and hands it apart from the features, so nothing can accidentally feed a model the thing it is meant to guess. The one rule to remember: a pipeline has one output, named right below the split.

You want to…	Write
predict a column that is already there	.Target("survived")
predict a column's return some rows further down the file	.Ahead("AAPL.Close", 5, AheadAs.Return)
predict the value itself some rows further down	.Ahead("close", 5)
predict the shares of a whole	.Distribution(bands, scaleBy: "chicks")
predict shares whose bands have an order	.Distribution(bands, scaleBy: "chicks", ordered: true)
predict a yes or no for each of several columns	.Labels(columns)
predict several free numbers	.Numbers(columns)

The rest of this page explains each one.

What the step looks like

using DeepSharp.Pipelines;

var prices = Pdd.Create()
    .ReadCsv("apple.csv")
    .Declare(schema => schema.Timestamp("Date").Number("AAPL.Close", "AAPL.Volume"))
    .OrderBy("Date")
    .SplitByTime("Date", train: 0.70, validation: 0.15, gap: 5)
    .Ahead("AAPL.Close", 5, AheadAs.Return)          // the answer: the return five days on
    .Drop("Date")                                    // what no model reads, before what learns
    .Normalise("AAPL.Close", "AAPL.Volume")
    .Build()
    .Run();

var train = prices.Batch(Part.Train);

Console.WriteLine($"{train.Features.Count} rows, {train.FeatureNames.Count} features, answers apart: {train.Labels is not null}");

It stands right below the split

A return is made from its column as it was read, so no step that changes that column may stand above it. Scale AAPL.Close first and the pipeline refuses the pair at the return:

Step 7, 'target.ahead': is a return on 'AAPL.Close' as it was read, and step 5, 'normalise', changes 'AAPL.Close' above it. Put the return above that step.

Named right below the split, the answer is valid whatever kind it is. That is why the course puts it there for a class as well, and what no model reads is dropped next, before what learns.

Target and Ahead are two questions, not one verb with a flag

• .Target("survived") says the answer is a column that is already there.
• .Ahead("close", 5, AheadAs.Return) makes the answer out of a column read further down the file — the price five days on, as a return rather than a level. The rows at the end that have no such future are kept apart rather than guessed at.

A column that is the answer is not also a feature

Leaving close in as a feature while asking for close five days on is fine and often right. Leaving the answer itself in is the leak that makes a model look perfect.

The handover knows which is which because the declaration said so.

One output, which can take several columns

A second output is refused when the pipeline is declared:

a pipeline has one output, and this is a second one; the first is 'target' at step 7.

What an output may do is name more than one column:

Output	For
Distribution	the shares of a whole
Labels	a yes or no each
Numbers	several free numbers

The handover gives each row those answers in the order the declaration named the columns. Two separate questions are two pipelines.

Shares can be in an order

A flock weighed in bands is one answer of shares, and the bands have an order that a share of a whole does not need: a prediction one band off is nearly right. .Distribution(bands, scaleBy: "chicks", ordered: true) says so.

• With it, the report may ask Metric.Emd, Metric.Kl and Metric.Rps of it, and a network may train by the earth mover's distance.
• Without it, they are refused.

A flock that arrives with fewer birds than were planned can name a remainder, one more answer made of what is left. Pipeline has the rules.

What the profile is for

'alive' hands a model the answer in another word, and 'class' says again what 'pclass' says. A profile of the source names columns like those before you declare them — and then the answer is not to declare them.

Next: Filling and scaling · Every output kind and what each refuses: Pipeline