A model does not have to begin here to run here. Three packages read what PyTorch, Keras or ONNX saved into the same network DeepSharp writes, so you can serve it, measure it, or train it further behind a pipeline and keep it with that pipeline as one file. The rule to remember: every number goes into the slot its path names, all of them at once or none, and whatever does not fit is refused by name rather than guessed.

The rest of this page explains each one: first how numbers go in, then each framework, then training an import further.

dotnet add package DeepSharp.Import.PyTorch
dotnet add package DeepSharp.Import.Keras
dotnet add package DeepSharp.Import.Onnx

Each is an IImporter, and importer.Read(stream) hands back a SavedNetwork, as reading a network's own file does. A SavedNetwork holds three things:

• the network;
• every slot holding the file's numbers;
• the loss it answers through.

A file of numbers alone is read into the network it is handed. A file that describes its network as well has that network built.

Every importer ends in one load. network.Load(entries) puts a tensor into each slot its path names, as PyTorch's load_state_dict does with strict=True:

• every slot of the network is written once;
• no slot the network does not have is written;
• each tensor has its slot's shape and holds finite numbers.

A batch normalisation's scale, shift and statistics go in as any slot's do:

using DeepSharp.Networks;
using DeepSharp.Tensors;

var stream = new RandomStream(42);
var network = new LayerStack(new Dense(2, 2, stream.Draw("initialise:0", 0, 0)), new BatchNorm(2));

network.Load(
[
    new SlotEntry("0.weight", Tensor.From(new Shape(2, 2), [1f, 0.5f, -0.5f, 2f]), "line 1"),
    new SlotEntry("0.bias", Tensor.From(new Shape(2), [0.25f, -0.25f]), "line 2"),
    new SlotEntry("1.weight", Tensor.From(new Shape(2), [1.5f, 0.5f]), "line 3"),
    new SlotEntry("1.bias", Tensor.From(new Shape(2), [0.125f, -0.125f]), "line 4"),
    new SlotEntry("1.running_mean", Tensor.From(new Shape(2), [0.5f, -1f]), "line 5"),
    new SlotEntry("1.running_var", Tensor.From(new Shape(2), [4f, 0.25f]), "line 6"),
]);

The third value of each entry is where its file holds it, in the file's own words — a tensor's name, a dataset's path, a line. A file of numbers has no lines and columns to name a fault by, so this is what a refusal points at.

The load puts in all of the entries or none. Whatever is wrong is refused at once with a SlotLoadException, whose Faults name each fault where its file holds it, and the network keeps what it held:

line 2: '0.bias' is a 2 slot here, and is written as 3.
line 3: '1.weight' holds finite numbers, and its value at 1 is NaN.
line 5: '1.bias' is written twice here, and only one of the two would be read.
line 7: '2.weight' is no slot of this network.
'1.running_var' is missing: every slot of the network is written.

A network's own file is read through the same load, so it refuses a missing, extra or misshapen slot in the same words.

Every name a file holds is shown in a refusal as Quoted() shows a file's text:

• a line break, a control character or a character that turns text round is written as its escape;
• a name past 200 characters is cut short with its length.

So a file cannot write a line of its own into a log.

A file PyTorch saved holds a network's numbers and not the network. So the reader is handed the network they belong to and the loss it answers through. That network is one of two kinds:

• a stack — in Keras's words or written by hand — its layers numbered as PyTorch's Sequential numbers its modules;
• a network written as code, its layers named as the PyTorch module named them.

using DeepSharp.Import.PyTorch;
using DeepSharp.Networks;
using DeepSharp.Tensors;

var titanic = new Sequential().Dense(16).Relu().Dense(1).Lower(new Shape(14), new RandomStream(7));

using var safetensors = File.OpenRead("titanic.safetensors");
var saved = new SafetensorsFile(titanic, new BinaryCrossEntropy()).Read(safetensors);

var images = new Sequential().Conv2D(4, new Window(3, 3)).Relu().Flatten().Dense(1)
    .Lower(new Shape(28, 28, 1), new RandomStream(7));

using var state = File.OpenRead("images.pt");                       // torch.save(model.state_dict(), "images.pt")
var read = new TorchSaveFile(images, new MeanSquaredError()) { Example = new Shape(28, 28, 1) }.Read(state);

SafetensorsFile reads a safetensors file and TorchSaveFile the archive torch.save has written since PyTorch 1.6. Both are a PyTorchFile, and put the same numbers into the same slots, bit for bit, whichever file they came in.

How PyTorch lays out each number is read off the layer that holds its slot, never off the number's shape — a square matrix turned round is as square as one that is not.

num_batches_tracked, which PyTorch keeps beside a batch normalisation's statistics, is left out by its name.

A linear layer or a convolution made with bias=False has no bias in the state dictionary. A normalisation made without its scale or its shift has no weight or no bias there.

The network handed to the reader says the same:

• new Beginning { WithoutBias = true } for a layer;
• new Affine { WithoutScale = true } for a normalisation.

The slots a layer lists decide what the reader expects of the file, so a layer without a bias expects none. A file that holds a number the layer has no slot for, or lacks one it has, is refused by name, as any file is. This needed no new code in the reader: it was held to files PyTorch made for rows, images, series and a volume, as safetensors and as .pt.

PyTorch lays a series, an image or a volume out channel by channel, and DeepSharp place by place. So the rows a flatten makes of them are in another order there, and every number the next linear layer — and any normalisation before it — keeps for them is put in its place here.

That takes knowing what the flatten is handed. Tell the reader one of two things:

• an Example the network takes, which the reader runs through it as zeros;
• Flattened, stating it by the flatten's path — for the network of images above, Flattened = new Dictionary<string, Shape> { ["2"] = new Shape(26, 26, 4) }.

Told neither, the reader refuses those numbers rather than guess. Read without turning, the flattened rows of a test network landed 0.043 away from PyTorch's answers; turned, within 3e−8.

A flatten is handed one of these:

Two axes are steps and channels only where the layer that walks axes nearest before the flatten walks one axis, with no reshape or linear layer between. Two axes a reshape made are rows laid out as steps by features, which PyTorch flattens as they stand, and the reader refuses them rather than turn rows that are not a series.

All of these are read from either file. A few rules go with them:

• No numbers, no tensor. A pooling, an adaptive pooling and a dropout hold no numbers, as ZeroPad holds none, so no tensor is for them.
• Average pooling counts the padding. PyTorch's AvgPool counts the border it was told to pad with, so the layer here is new AvgPool2D(window) { CountsPadding = true }. A max pooling pads with minus infinity there and sees only values here, which is the same.
• Keeping the axes. An adaptive pooling to one place is a global pooling with KeepsAxes = true where the module keeps its axes, or without it where a Flatten after it is left out.
• What a state dictionary does not hold. It holds no stride, dilation, padding mode or ceil_mode. The network handed to the reader says them; none of those can be seen in a file of numbers, and none is detected.
• Kernel layout follows the layer. A grouped convolution's rows count the values of a window for the channels of one group. A transposed convolution's rows are the channels it is handed, each holding the window's places by the channels it makes.

• Activations hold no numbers and need no tensor.
• PReLU's shares go into PRelu's weight as they are: one, or one for each channel, and after a flatten one for each feature in the order the flatten makes.
• Embedding's vectors go into Embedding's weight, a row for each category, which is the row PyTorch looks up by the category's place. Hand the network here the category's ones and noughts where PyTorch was handed its place.
• GroupNorm, InstanceNorm with affine=True and RMSNorm keep their scale and shift as the normalisations here keep theirs.
• The periodic embedding of rtdl_num_embeddings is no module of PyTorch's, so its numbers go in by hand.

A network written as code shows the layers it holds, each with its path — layer.HeldLayers() lists them, as Slots() lists the numbers — but keeps to itself the order its forward pass runs them in.

Its weights and kernels are turned as a stack's are. Whether one of its linear layers or normalisations reads rows made of images is never guessed: wherever such rows could reach one, it is read only as Flattened states it by that layer's path, and refused otherwise. A layer of a kind the reader does not know has none of its numbers read.

• Held in 16 bits — half precision or bfloat16 — numbers are widened exactly.
• Held in 64 bits, they are rounded to the nearest 32-bit float, as PyTorch rounds them.
• Whole numbers and truth values are refused.

A safetensors file is read by Onnxify.Safetensors, a port of safetensors' own reader, so a file that reader refuses is refused here in its words — of twenty written wrongly on purpose, the eighteen it refuses.

A file whose note names another framework's layout than "format": "pt" is refused too. One with no note is read as PyTorch's, since safetensors.torch.save_file and save_model write none unless handed one.

The pickle torch.save writes the state into is a program, whose instructions name functions and call them, so no pickle library reads it. An interpreter written here carries out only the instructions PyTorch's own weights-only reader, torch.load(weights_only=True), carries out.

It builds nothing a file names but what a state dictionary is made of:

• an OrderedDict;
• a tensor rebuilt by torch._utils._rebuild_tensor_v2 or _rebuild_parameter;
• the storages of a kind of number.

Every other name is refused where the file names it, before anything is looked up, built or run.

These are all read: tensors viewing their storage from any offset and with any strides, several sharing one storage, a file a big-endian machine wrote, and forty layers in a row, whose pickle numbers its memo past 255.

Refused, each by name:

• a pickle that is no dictionary of tensors by name — a checkpoint that keeps the state under a key of its own is read once model.state_dict() is saved on its own;
• a torch.Size, bytes or a Counter, which PyTorch's reader allows and a state dictionary never holds;
• the format before PyTorch 1.6, and a TorchScript archive;
• a tensor saved as a negated or conjugated view — save the tensor itself — and one that repeats its storage's numbers.

Of 49 files written wrongly or with hostile intent on purpose, each handed to PyTorch's own reader first, every one it refuses is refused here too. Of those it reads, six are refused here: a torch.Size, bytes, a Counter, the format before 1.6, a checkpoint under a key of its own and a negated view. What a file can make the reader do is bounded by the file and the network — Security says how.

• Titanic. The Titanic network PyTorch trained on the pipeline's training rows answers a man of 22 in third class within one rounding of a single-precision number of PyTorch's chance, and all 135 test passengers within five, written in Keras's words or as code.
• With a batch normalisation, within ten, since PyTorch folds the normalisation into one scale and one shift where here a feature is normalised and then scaled.
• Series, images and volumes. A network of each kind — convolutions with stride and padding, poolings, an adaptive pooling and a dropout of channels, a batch normalisation, a flatten and linear layers — answers within a hundred-thousandth of PyTorch's output, from a safetensors file and from a .pt file, which put the same bits into the same slots.

The files and what PyTorch answered are kept with the tests, with the scripts that made them.

using DeepSharp.Import.Keras;

using var file = File.OpenRead("titanic.keras");
var saved = new KerasFile().Read(file);

A Keras file describes its network as well as holding its numbers, so the reader builds the network from the description. Each layer is written in Keras's words and lowered as any description in those words is:

• a batch normalisation keeps Keras's epsilon and the complement of its momentum;
• a window is padded as TensorFlow's 'same';
• the activation a Keras layer carries becomes a layer of its own after it.

The numbers need no turning — Keras lays them out as the slots here keep them. Each layer's numbers are found by the name the file gives them, never by where they stand: Keras files them under the layer's class, layers/dense, while the layer is dense_2.

The loss is the one the model was compiled with: a binary or a categorical cross-entropy, a mean squared error, the mean absolute error, the Huber loss, the Poisson loss on rates, the hinge or the binary focal cross-entropy. A cross-entropy's label_smoothing is read as its smoothing.

The network ends where that loss takes over:

• a last sigmoid before a binary cross-entropy, or a last softmax before a categorical one, trained on the chances it gives, is lifted into the loss, as Keras's habit is;
• a model trained on its logits is read as it stands;
• one whose end and loss disagree is refused at that layer.

A Sequential model, or a Functional one whose layers each take the layers they name, working in single precision, of these layers:

The activation words read are linear, relu, tanh, sigmoid, gelu, silu, swish, mish, leaky_relu, elu, selu and softmax.

• Strides. A pooling that says no strides strides by its pool size, as Keras's does. A whole number stands for every axis.
• Window settings. A stride of its own for each axis, a dilated window and a convolution's groups are read as the window's own settings.
• Causal. A convolution along a series may pad as 'causal'.
• Keeping the axes. keepdims of a global pooling is KeepsAxes.
• Depthwise kernel. Keras's own layout: the channels it is handed by the channels it makes of each.
• Transposed kernel. Keras's layout the other way round. A transposed convolution padded as 'same' states no border in the file, so the border is worked out from the sizes Keras itself makes.
• Separable convolution. Two layers here. Keras adds its one bias after both, so the first has none and the second carries it.
• Zero padding. ZeroPadding1D writes its two ends as the pair itself, where ZeroPadding2D and ZeroPadding3D write a number or a pair for each axis.

• LeakyReLU takes Keras's negative_slope, and the activation word leaky_relu Keras's two tenths.
• PReLU's shares go in as Keras writes them, a length of one for every axis they are shared over: one share for each feature of a row, or for each channel of an image whose shared_axes are its places.
• GroupNormalization of a group for every channel is an instance normalisation. Keras writes that as -1 before the model is built and as the count of its channels after.
• RMSNormalization is the root-mean-square normalisation. A LayerNormalization with rms_scaling is refused: Keras works that scale out from the variance rather than the root mean square, and says so as it deprecates it.
• Embedding is read as the first layer, over one category's place per example. The network here is handed that category's ones and noughts instead, so its input is as wide as there are categories. One that masks the place nought, or keeps its vectors as low-rank factors, is refused.

No bias, no scale, no shift. A layer made with use_bias=False, and a normalisation made with scale=False or center=False, is read as the layer without it. The file holds no numbers for what is not there, and a description that says a bias the file has no numbers for, or the other way round, is refused at the number.

Regularizers. A layer's regularizers of L1, L2 and L1L2 are read as the penalty its numbers carry, with Keras's own default where the class names no number. They are kernel_ and bias_, gamma_ and beta_ for a normalisation, and depthwise_ and pointwise_ for the separable kind.

MaxNorm. A MaxNorm constraint is read as the bound for each unit, but only over the axes a unit's numbers lie along:

• a dense kernel's first axis;
• a convolution's kernel over everything but its channels out;
• a depthwise kernel over the window's axes;
• a bias's only axis.

A bound over any other axis is not the bound the layer applies, and a different bound is not read as the same one.

Held layers. trainable=False is read as the layer held still. Networks says what each of these does.

Initialisers are not read: a model that was read starts from its own numbers, so there is nothing for an initialiser to start.

What no network here is built of is refused at the layer that says it, every such layer at once:

• any other kind of layer or activation — the refusal names every kind that is read;
• channels first;
• a normalisation over another axis than the last;
• an activity_regularizer, which penalises what a layer makes rather than its numbers;
• a regularizer other than L1, L2 and L1L2;
• a constraint other than MaxNorm, a MaxNorm over other axes than a unit's numbers lie along, and a bound on a transposed convolution's kernel;
• a transposed convolution that pads as 'same' and also states how many places to keep at the end, since Keras reads the first of those and leaves the second;
• a model of another kind than a Sequential, in another precision, or saved without its loss.

• Titanic. The networks sample's Titanic network, trained in Keras 3 on the pipeline's training rows and saved both ways, answers the 135 test passengers within five roundings of Keras's chances — 68 of them to the bit — and the two passengers the sample serves within two, from either file.
• Every kind. A network of every kind the reader builds answers within six.
• Series, images and volumes. A network over each, built of the convolutions, poolings, global poolings and spatial dropouts, answers within a hundred-thousandth of Keras's output, from the archive and from the HDF5 file.

Where a padded average pooling strides, Keras on its torch backend does not follow TensorFlow's rule. It averaged the end of the series 1, 2, 3, 4 in a window of three at a stride of two to 3.67, where TensorFlow's rule gives 3.5. So the reference for that case is TensorFlow's rule worked out by hand, which is the rule here.

using DeepSharp.Import.Onnx;
using DeepSharp.Networks;

using var graph = File.OpenRead("titanic.onnx");
var saved = new OnnxFile(new BinaryCrossEntropy()).Read(graph);    // a graph names no loss, so it is handed one

Three writers were measured and are read:

• PyTorch's torch.onnx.export, by its default exporter and by the TorchScript one before it;
• Keras's own model.export(format="onnx");
• tf2onnx, which is how a TensorFlow SavedModel reaches ONNX — python -m tf2onnx.convert --saved-model.

Read the graph from its file: PyTorch's default exporter keeps larger numbers in a file beside the graph, and they are read from beside the graph's own file.

The graph is read once and lowered onto the layers it is, never kept beside them.

Asked for an opset that has no operator for a word, PyTorch's exporters write it out of several. Those are read as the one layer each spells:

A chain is read so only where every number in it is the word's own and nothing outside it takes a value it makes. Otherwise its nodes are read one at a time, and refused where they are no layer. A GroupNormalization of opsets 18 to 20 scales and shifts each group rather than each channel, and is refused.

Layouts are what each node declares, never what the numbers look like:

• a weight matrix is turned round where its Gemm says it is written outputs by inputs, as PyTorch writes one;
• a kernel is written channels out, channels in, then the window's axes, as ONNX defines it;
• a series, an image or a volume flattened channel by channel there is flattened place by place here, so the numbers along such a row are turned from the one order to the other.

A series an LSTM or a GRU reads is taken step by batch by channel by the Transpose that first takes it, and what they give is put back by the squeezes, transposes, gathers and reshapes PyTorch's exporter writes; none of those is a layer here.

Series, images and volumes go into the network, and come out, with their channels last. A graph that takes them channels last, as tf2onnx's does, says so with the Transpose that moves them after the batch before its convolutions. That Transpose, and the one moving them back before its flatten, are nothing here.

Numbers of half precision or bfloat16 are widened exactly, numbers of double precision rounded to the nearest single one.

• Window settings. A ceil_mode, dilations, a stride of its own for each axis and a group are read as the window's own settings.
• Transposed convolution. A ConvTranspose is read with its output_padding as the places kept at the end. A node that says the output_shape a transposed convolution makes is refused, naming output_padding instead.
• Borders. A border written out one side at a time is read as TensorFlow's 'same' where it is the border 'same' gives along each axis, and as a stated border otherwise. SAME_UPPER is 'same', and SAME_LOWER is read only where it gives the same border.
• Average pooling counts the border it pads with when count_include_pad is set, and leaves it out otherwise.
• Pad. In constant mode it is read with the value the graph gives it. In reflect, edge or wrap mode it is a padding that mirrors, repeats or goes round (wrap from opset 19, where ONNX has it). A Pad of any other mode than those four, or that pads the batch's own axis or its channels, is refused saying so.
• No bias. A Conv, a ConvTranspose and a LayerNormalization that name no bias are read as the layer without one, which is how PyTorch exports a layer made without it.

At its node, and every such node at once:

• a node taking values no node made, or several that are no Add of two or Concat;
• an LSTM or a GRU given the lengths of its series, a state the graph works out, peepholes, other activations, a clip, a batch-first layout, a direction back alone, a GRU that resets before the product; one whose steps and last states are both taken; and the last states of a stack of layers picked out by Gather and Concat;
• an Attention given a mask, a scale of its own, a softcap, its scores handed on, or a past of keys;
• any other operator;
• a max pooling's storage_order, or its second output when the graph uses it;
• a Cast into another type;
• a Transpose other than the two above;
• an input of whole numbers — the places a Gather looks an embedding up by, whose vectors go in through the PyTorch or the Keras reader instead.

tf2onnx writes a batch normalisation as a Mul, and a flatten for batches of any length through a side graph of Shape, Gather, Slice and Concat. Neither is a layer here. So a network with a batch normalisation does not reach here through tf2onnx, and one that flattens images does when its SavedModel is exported for a batch of a fixed length.

• Titanic from PyTorch. Exported both ways, it answers the README's passenger within one rounding of PyTorch's chance and all 135 test passengers within five.
• Titanic from Keras. Exported by Keras or converted by tf2onnx 1.17.0 from TensorFlow 2.21.0's SavedModel, it answers the 135 within five of Keras's chances and puts the very bits its .keras archive does into the slots.
• Images. A network of convolutions, batch normalisations and a flatten gives each image PyTorch's output within 4.5e−8. One of 'same' and strided convolutions that tf2onnx converted gives each TensorFlow's within 2.4e−7.
• Series, images and volumes. A network over each, exported by PyTorch's default exporter and by the TorchScript one and each ending in a global max pooling or a global average pooling, answers within a hundred-thousandth of PyTorch's output. The graphs are held to ONNX Runtime too.

The default exporter writes an adaptive pooling to one place as a ReduceMean or ReduceMax, and round a series an Unsqueeze and a Squeeze. That exact pattern is read as the global pooling it is, and any other Unsqueeze, Squeeze or reduction is refused. Only the TorchScript exporter writes GlobalAveragePool, and GlobalMaxPool only where the spatial lengths are left open.

An import is a network and its loss, and records no fit of a pipeline — its TrainedOn is nothing. So it stands behind a pipeline as any network does: compiled with an optimizer and its loss, and fitted here on the rows the pipeline prepares, for one epoch or for as many as training it further takes. That fit is what the one file records, and the file is read back beside that fit alone:

using DeepSharp.Import.Keras;
using DeepSharp.Learners.Networks;
using DeepSharp.Networks;
using DeepSharp.Pipelines;

var prepared = Pdd.Create()
    .ReadCsv("titanic.csv")
    .Declare(schema => schema.Integer("survived", "sibsp", "parch").Category("pclass", "sex").Optional("age", ColumnKind.Number).Number("fare"))
    .SplitStratified("survived", train: 0.70, validation: 0.15)
    .Target("survived")
    .FillMissing("age", With.Median)
    .EncodeCategories()
    .Normalise("age", "fare", "sibsp", "parch")      // a network takes them between -1 and 1, which is the default
    .Build()
    .Run();

using var file = File.OpenRead("titanic.keras");
var saved = new KerasFile().Read(file);

var trained = saved.Network.Compile(new Adam(0.001), saved.Loss).Fit(prepared, new FitOptions(seed: 7) { Epochs = 1 });
File.WriteAllText("titanic.network.json", trained.ToJson());

This is the door to take when the network is not yours to write. A declaration names a network by its words, while the numbers of an imported one come from a file, so the pipeline is only run, and the network it already is, is fitted behind it. What comes out is the same TrainedNetwork a declared run gives, kept with its pipeline as one file.

A model read from elsewhere learns on everywhere, at one rate, unless a layer is held. A Keras layer saved with trainable=False arrives already held, and any layer can be held by hand. So the layers that carry what was learned stay as they are while the ones after them are trained here.

The held layers are part of the run, so a checkpoint records them and refuses a run that goes on with others held.

using DeepSharp.Import.Keras;
using DeepSharp.Networks;

using var file = File.OpenRead("titanic.keras");
var saved = new KerasFile().Read(file);

// The first dense layer keeps what Keras learned; every layer after it is trained here.
saved.Network.HeldLayers().Select(each => each.Layer).OfType<Dense>().First().Hold();

var compiled = saved.Network.Compile(new Adam(0.001), saved.Loss);

The networks sample's Titanic network Keras trained, fitted one epoch further behind this pipeline, was read back from its file holding the same numbers to the bit and giving the passengers the same chances.

An import answers rows handed to it as a tensor of features, saved.Network.Predict(features, saved.Loss, engine), on any engine — the TorchSharp backend among them.

A reader of another framework's files implements IImporter. Read(stream) hands back a SavedNetwork, and the numbers go in through network.Load, each as a SlotEntry holding three things:

• the path of its slot;
• the tensor, laid out as the slot keeps it;
• where its file holds it.

The work is split:

• The reader's: turning a number into the layout its slot keeps, before it hands it over, and building the network when its file describes one.
• The load's: refusing what does not fit, in the same words for every importer.