Saving and replaying - xkqg/DeepSharp GitHub Wiki
Step 10 β Saving and replaying
In short
A pipeline is saved as one file with two halves: the declaration, which is what you wrote, and the fitted half, which is what the fit learned. Reading it back needs no data, and replaying it walks the same steps over new rows with the numbers the training rows produced. The one rule to remember: replaying is not re-fitting β nothing is learned again.
| You want to⦠| Write |
|---|---|
| write both halves as text | prepared.ToJson() |
| read both halves back, anywhere, with no data | PreparedData.FromJson(text, StepCatalog.BuiltIn()) |
| read the person's half alone | PipelineDeclaration.FromJson(text, StepCatalog.BuiltIn()) |
| see which version of the file this library writes | PipelineDeclaration.Version |
| walk the fitted steps over rows nobody had seen | prepared.Replay(rows) |
| hand over rows without an answer, to predict on | prepared.Served(rows) |
| put predictions back in your own units | prepared.BackToOriginal(predictions) |
| measure a window of served rows against the training rows the file keeps | prepared.Watched(window) |
The rest of this page explains each one.
Two halves, two authors
The two halves have two different authors, and keeping them apart is what lets:
- the same declaration be fitted again on fresh data;
- two runs be compared by diffing the declaration alone;
- a serving process load the file without ever knowing a builder existed.
using DeepSharp.Pipelines;
var prepared = Pdd.Create()
.ReadCsv("titanic.csv")
.Declare(schema => schema.Integer("survived").Optional("age", ColumnKind.Number).Number("fare"))
.SplitStratified("survived", train: 0.70, validation: 0.15)
.Target("survived")
.FillMissing(fill => fill.Median("age"))
.Normalise("age", "fare")
.Build()
.Run();
var text = prepared.ToJson(); // both halves
var again = PreparedData.FromJson(text, StepCatalog.BuiltIn()); // anywhere, no data needed
var declared = PipelineDeclaration.FromJson(text, StepCatalog.BuiltIn()); // the person's half alone
Console.WriteLine($"{declared.Steps.Count} steps, version {PipelineDeclaration.Version}");
A fit is tied to the steps it was fitted behind
Every fitted entry carries a key made from its own step and every step above it, so a fit is never used under steps that changed after it was learned.
Read back by position instead, a fit spliced under another declaration once served a price of 135.7 where 0.9048 was meant β which is why it is keyed by content and not by place.
Replaying is not re-fitting
Replay(rows)walks the same steps over rows nobody had seen, with the numbers the training rows produced and nothing fitted again. It puts the rows in order and drops the warm-up rows exactly as the run did.Served(rows)hands rows over without an answer, because a served row is the question, and says which handed-in row each one is so a prediction finds its way back.BackToOriginal(β¦)puts the predictions back in your own units.
A file from an older DeepSharp still reads
The file names its version, and each verb the version from which it means what it says now. A file newer than the library is refused whole rather than half-understood.
A model's own file carries its pipeline's text exactly as that text was written, and is tied to it by a digest over that text. So the version the text names never changes, and a model trained behind an older pipeline keeps loading and keeps serving.
Two pipelines are compared fit for fit by a second digest, and that is the one that leaves the version out.
This library writes the eleventh version, which adds watch.reference; 0.11.0 refuses a file of it whole, naming the
version. A file names the version of the library that wrote it, never the oldest one its steps would need, so a pipeline
that declares nothing new is still written as the eleventh. A pipeline fitted by 0.11.0 keeps reading and its model keeps
serving, but it keeps no training rows, so a window of served rows has nothing to be measured against until the pipeline
is fitted again with the watch and the model trained behind that fit β see Measuring and drawing.
A network's file says only what it holds
The parts of a network's own file β the network, a run's training β have a version of their own, and each is written as the oldest one that says what it holds:
| What the part holds | Written as | An older library |
|---|---|---|
| a layer without a bias, a padding that fills from the example, a number with a penalty, a bound or a hold, an optimizer with one of the options added in 0.10.0, or a run with a clip of each value, a window of batches or an average | the fifth version | 0.9.0 refuses it by its number |
| a cross-entropy that softens its labels, or a run judged by the average of its weights | the sixth version | 0.10.0 refuses it by its number |
| a new kind of layer or loss | the second version | refuses it by its name |
| everything else | as it was | reads it |
So a file an earlier release wrote is read, written again and served as it always was.
A model from another library says more
A trainer from ML.NET produces that library's own archive, which only it opens. So the file carrying it also names the version of ML.NET that wrote it and the processor it was written on β a reader that cannot open the archive can then say which package and which version would.
Its bytes are never what ties the file to anything: they carry the clock they were saved at, so two saves of one model differ while the model does not.