1. Overview
2. Training Architecture
3. Development Environment
4. Trainer API
5. Dataset Preparation
6. DataLoader Integration
7. Model Construction
8. Loss Functions
9. Optimizer Configuration
10. Complete Training Example
11. Validation and Evaluation
12. Training History
13. Callbacks
14. Early Stopping
15. Model Checkpointing
16. Learning-Rate Scheduling
17. Gradient Clipping
18. Experiment Tracking
19. Reproducibility
20. Performance Optimization
21. Testing and Validation
22. Troubleshooting
23. Developer References
24. Next Steps

Model training is the process of updating neural network parameters using examples, objective functions, and optimization algorithms.

EveryO provides a Trainer abstraction to coordinate training and evaluation.

A reliable training workflow should support:

• Correct gradient computation.
• Stable parameter updates.
• Data separation.
• Metric monitoring.
• Reproducibility.
• Checkpoint management.
• Error detection.
• Performance measurement.

                  TRAINING DATA
                       |
                       v
                    DataLoader
                       |
                       v
                  Neural Network
                       |
                       v
                    Predictions
                       |
                       v
                   Loss Function
                       |
                       v
                 Backpropagation
                       |
                       v
                     Optimizer
                       |
                       v
                Updated Parameters
                       |
                       v
                  Next Mini-Batch

           Validation / Evaluation
                       |
                       v
                Metrics & History
                       |
                       v
                    Callbacks

1. Initialize the model.
2. Prepare datasets.
3. Configure loss.
4. Configure optimizer.
5. Initialize trainer.
6. Execute training epochs.
7. Evaluate validation data.
8. Record metrics.
9. Execute callbacks.
10. Save results.

git clone https://github.com/krishanth7/EveryO.git
cd EveryO

python -m pip install -e .

python -m pip install -e ".[dev]"

import everyo as eo

print("EveryO version:", eo.__version__)

EveryO declares:

• Python 3.9 or newer.
• NumPy 1.22 or newer.
• Matplotlib 3.5 or newer.

Optional integrations have additional requirements.

The Trainer class coordinates model training, evaluation, and inference.

Trainer(
    model,
    optimizer,
    loss_fn,
    *,
    metrics=(),
    device="cpu",
    gradient_clip=None
)

import everyo as eo

model = eo.Sequential(
    eo.Linear(4, 8, seed=0),
    eo.ReLU(),
    eo.Linear(8, 2, seed=1)
)

optimizer = eo.Adam(
    model.parameters(),
    lr=0.001
)

trainer = eo.Trainer(
    model,
    optimizer,
    eo.CrossEntropyLoss(),
    metrics=["accuracy"],
    device="cpu"
)

The inspected fit() signature is:

trainer.fit(
    train_loader,
    *,
    epochs=10,
    validation_loader=None,
    callbacks=(),
    verbose=True,
    log_every=1
)

fit() returns an EveryO History object containing recorded epoch metrics.

Good training results depend on reliable input data.

• Correct feature dimensions.
• Valid target values.
• Consistent data types.
• Appropriate preprocessing.
• Training and validation separation.
• Protection against data leakage.

import numpy as np

rng = np.random.default_rng(42)

features = rng.normal(
    size=(200, 4)
).astype("float32")

targets = (
    features.sum(axis=1) > 0
).astype("int64")

This synthetic dataset defines a binary classification task.

train_x = features[:160]
train_y = targets[:160]

val_x = features[160:]
val_y = targets[160:]

For this independently generated synthetic dataset, the example uses a straightforward split.

Real-world datasets should use an appropriate splitting strategy, including temporal or group-aware separation when needed.

EveryO exports ArrayDataset and DataLoader.

import everyo as eo

train_dataset = eo.ArrayDataset(
    train_x,
    train_y
)

val_dataset = eo.ArrayDataset(
    val_x,
    val_y
)

train_loader = eo.DataLoader(
    train_dataset,
    batch_size=16
)

val_loader = eo.DataLoader(
    val_dataset,
    batch_size=16
)

Batch size determines how many examples are processed together.

For:

[ N=160 ]

and:

[ B=16 ]

the number of full batches is:

[ 160/16=10 ]

• Batch construction.
• Dataset iteration.
• Input and target pairing.
• Optional ordering controls.
• Consistent batch shapes.

model = eo.Sequential(
    eo.Linear(4, 8, seed=0),
    eo.ReLU(),
    eo.Linear(8, 2, seed=1)
)

Input Features: 4
       |
       v
Linear Layer: 8
       |
       v
ReLU Activation
       |
       v
Linear Layer: 2
       |
       v
Class Logits

First layer:

[ 4\times8+8=40 ]

Second layer:

[ 8\times2+2=18 ]

Total:

[ P=58 ]

• Input feature count.
• Output class count.
• Hidden-layer size.
• Activation selection.
• Parameter initialization.
• Computational cost.

Loss functions quantify model prediction error.

loss_fn = eo.CrossEntropyLoss()

loss_fn = eo.MSELoss()

Use a loss function that matches the output representation and target format.

For example, multiclass cross-entropy commonly expects logits and class-index targets.

Do not apply softmax unnecessarily before a loss implementation that already handles logits.

EveryO exports SGD and Adam.

optimizer = eo.SGD(
    model.parameters(),
    lr=0.01
)

optimizer = eo.Adam(
    model.parameters(),
    lr=0.001
)

\theta_t-\eta\nabla_\theta L ]

The actual update depends on the selected optimizer.

• Learning rate.
• Momentum.
• Weight decay.
• Gradient magnitude.
• Numerical precision.
• Optimizer state.

Related: Optimization

Objective: Train a binary classifier using EveryO's built-in training engine.

import numpy as np
import everyo as eo

# ---------------------------------
# 1. Generate synthetic data
# ---------------------------------

rng = np.random.default_rng(42)

x = rng.normal(
    size=(200, 4)
).astype("float32")

y = (
    x.sum(axis=1) > 0
).astype("int64")

# ---------------------------------
# 2. Split the dataset
# ---------------------------------

train_x = x[:160]
train_y = y[:160]

val_x = x[160:]
val_y = y[160:]

# ---------------------------------
# 3. Create datasets
# ---------------------------------

train_dataset = eo.ArrayDataset(
    train_x,
    train_y
)

val_dataset = eo.ArrayDataset(
    val_x,
    val_y
)

# ---------------------------------
# 4. Create data loaders
# ---------------------------------

train_loader = eo.DataLoader(
    train_dataset,
    batch_size=16
)

val_loader = eo.DataLoader(
    val_dataset,
    batch_size=16
)

# ---------------------------------
# 5. Create model
# ---------------------------------

model = eo.Sequential(
    eo.Linear(4, 8, seed=0),
    eo.ReLU(),
    eo.Linear(8, 2, seed=1)
)

# ---------------------------------
# 6. Configure optimizer
# ---------------------------------

optimizer = eo.Adam(
    model.parameters(),
    lr=0.01
)

# ---------------------------------
# 7. Initialize trainer
# ---------------------------------

trainer = eo.Trainer(
    model,
    optimizer,
    eo.CrossEntropyLoss(),
    metrics=["accuracy"]
)

# ---------------------------------
# 8. Train model
# ---------------------------------

history = trainer.fit(
    train_loader,
    epochs=20,
    validation_loader=val_loader,
    verbose=False
)

# ---------------------------------
# 9. Evaluate model
# ---------------------------------

results = trainer.evaluate(val_loader)

print("Evaluation:", results)

# ---------------------------------
# 10. Inspect training history
# ---------------------------------

print("Completed epochs:", history.epochs)
print("Available metrics:", history.keys)

The script should:

1. Generate synthetic training data.
2. Construct a classifier.
3. Execute training epochs.
4. Evaluate validation data.
5. Return training history.

Numerical results are not fixed and should be measured in the actual environment.

Validation measures model behavior on data outside the training subset.

results = trainer.evaluate(val_loader)

The trainer supports metric names and callable metrics.

Example:

metrics=["accuracy"]

Evaluation returns a dictionary of loss and metric values.

• Keep validation data separate.
• Avoid preprocessing leakage.
• Monitor both loss and task-specific metrics.
• Compare against simple baselines.
• Reserve an independent test set for final assessment.

EveryO's History object records per-epoch training information.

print(history.epochs)
print(history.keys)
print(history.last())

loss_values = history["loss"]

print(loss_values)

best_epoch = history.best(
    key="val_loss",
    mode="min"
)

print(best_epoch)

history.to_json("runs/history.json")

history.to_csv("runs/history.csv")

Callbacks observe or influence the training process.

The inspected base class provides:

on_train_begin
      |
      v
on_epoch_begin
      |
      v
on_batch_begin
      |
      v
on_batch_end
      |
      v
on_epoch_end
      |
      v
on_train_end

Batch hooks repeat for each batch, while epoch hooks repeat for each epoch.

import everyo as eo

class EpochReporter(eo.Callback):

    def on_epoch_end(self, trainer, epoch, logs):
        print(
            f"Epoch {epoch}:",
            logs
        )

Use it with:

history = trainer.fit(
    train_loader,
    epochs=10,
    callbacks=[EpochReporter()]
)

Early stopping can end training when a monitored metric stops improving.

EarlyStopping(
    monitor="val_loss",
    *,
    patience=5,
    min_delta=0.0,
    mode="min",
    restore_best_weights=True,
    verbose=True
)

early_stopping = eo.EarlyStopping(
    monitor="val_loss",
    patience=5,
    mode="min",
    restore_best_weights=True
)

history = trainer.fit(
    train_loader,
    epochs=100,
    validation_loader=val_loader,
    callbacks=[early_stopping]
)

The inspected implementation stops when the non-improvement counter exceeds patience.

Restoring the best weights means the final model may correspond to an earlier epoch, not the last recorded epoch.

Checkpoints preserve model state for later use.

Checkpointing can support:

• Model recovery.
• Experiment comparison.
• Best-model selection.
• Reproducible evaluation.
• Training continuation.

eo.ModelCheckpoint

import inspect
import everyo as eo

print(
    inspect.signature(eo.ModelCheckpoint)
)

Use the inspected signature when configuring checkpoint paths, monitored metrics, or save conditions.

• Model architecture.
• Parameter state.
• Optimizer configuration.
• Training epoch.
• Validation metrics.
• Framework version.
• Dataset identifier.
• Experiment configuration.

A learning-rate scheduler changes the optimizer's learning rate during training.

eo.LearningRateScheduler

Common strategies include:

• Step decay.
• Exponential decay.
• Cosine scheduling.
• Warmup.
• Metric-dependent adjustment.

Availability of individual strategies depends on the callback implementation and supplied schedule function.

import inspect
import everyo as eo

print(
    inspect.signature(
        eo.LearningRateScheduler
    )
)

Track the learning rate alongside:

• Training loss.
• Validation loss.
• Gradient norms.
• Training duration.

Gradient clipping limits the magnitude of gradients before optimizer updates.

The inspected Trainer constructor supports:

gradient_clip=1.0

trainer = eo.Trainer(
    model,
    optimizer,
    eo.CrossEntropyLoss(),
    metrics=["accuracy"],
    gradient_clip=1.0
)

For gradient vector (g) and threshold (c):

g\cdot \min\left( 1,\frac{c}{|g|_2} \right) ]

for nonzero gradient norm.

Clipping can help with excessively large gradients but does not replace proper numerical validation or learning-rate selection.

Experiment tracking provides a record of model development.

experiments/
 |
 +-- run-001/
 |    +-- config.json
 |    +-- history.json
 |    +-- history.csv
 |    +-- model-checkpoint
 |
 +-- run-002/
      +-- config.json
      +-- history.json
      +-- history.csv
      +-- model-checkpoint

history.to_json(
    "experiments/run-001/history.json"
)

history.to_csv(
    "experiments/run-001/history.csv"
)

Reproducible training requires controlling relevant sources of variation.

• Model initialization.
• Dataset generation.
• Data ordering.
• Dependency versions.
• Backend behavior.
• Numerical precision.
• Optimizer configuration.
• Hardware.

model = eo.Sequential(
    eo.Linear(4, 8, seed=42),
    eo.ReLU(),
    eo.Linear(8, 2, seed=43)
)

• Record all relevant seeds.
• Save configuration.
• Pin dependencies.
• Track the source commit.
• Preserve evaluation datasets.
• Document hardware.

A seed alone does not guarantee identical results across all platforms and environments.

Training performance depends on computation, memory, and data processing.

• Batch size.
• Model complexity.
• Tensor dimensions.
• Backend choice.
• Numerical precision.
• Data loading.
• Intermediate allocations.

The CPU backend is the foundational execution path.

Optional acceleration requires supported hardware and a compatible implementation.

Measure:

1. Epoch duration.
2. Batch throughput.
3. Peak memory.
4. Training loss.
5. Validation metrics.
6. Backend configuration.

Performance improvements must preserve acceptable numerical correctness.

Training code should be tested before being used for significant experiments.

python -m pytest

• Dataset shapes.
• DataLoader output.
• Model forward pass.
• Loss calculation.
• Gradient generation.
• Optimizer updates.
• Training history.
• Callback execution.
• Validation metrics.
• Checkpoint behavior.
• Early stopping.
• Error handling.

import numpy as np
import everyo as eo

x = np.random.default_rng(0).normal(
    size=(32, 4)
).astype("float32")

y = (
    x.sum(axis=1) > 0
).astype("int64")

loader = eo.DataLoader(
    eo.ArrayDataset(x, y),
    batch_size=8
)

model = eo.Sequential(
    eo.Linear(4, 8, seed=0),
    eo.ReLU(),
    eo.Linear(8, 2, seed=1)
)

trainer = eo.Trainer(
    model,
    eo.Adam(model.parameters(), lr=0.01),
    eo.CrossEntropyLoss(),
    metrics=["accuracy"]
)

history = trainer.fit(
    loader,
    epochs=2,
    verbose=False
)

assert history.epochs == 2

Continue with the following EveryO Wiki pages.

Wiki links become active after their corresponding pages are created.