Experiment Logging

In order to keep track of all exectuted exeriments, the training history, and model evaluation are saved automatically into a log file. As soon as the model is merged into the master branch, a wiki entry is created automatically to visualize the training results.

Logging traing results

Use the utils.save_model.save_model function to save the model:

e.g.

from utils.save_model import save_model

model_id = save_model(model=model, history=result.history, name='inception', filename='inception-test.h5', description='Model description', version='1.0' )

Where the result object is the one returned when exectuting model.fit. When training the same model with different parameters, the model name should remain the same and there will be a wiki entry containing all the different versions of the model results.

The function will return the model id which can be used to store the test results.

Adding model evaluation

After training the model, the test results can also be logged, using the utils.save_model.model_set function.

from utils.save_model import model_set

pred=model.predict_generator(test_generator, steps=STEP_SIZE_TEST, verbose=1)
pred_bool = (pred >= 0.5)
y_pred = np.array(pred_bool, dtype=int)

dtest = data_test.to_numpy()
y_true = np.array(dtest[:,1:15], dtype=int)
report = classification_report(y_true, y_pred, target_names=list(data_test.columns[1:15]))
model_id = model_set(model_id, 'classification_report', report)

And for testing scores and accuracy:

score, acc = model.evaluate_generator(test_generator, steps=STEP_SIZE_TEST)
model_id = model_set(model_id, 'test', (score, acc))