What results to display - martynwinn/map-recognition GitHub Wiki

Global quantities

Such as overall metrics obtained from Keras. Currently we use the defaults which are loss and accuracy. Want to know how these vary with hyper-parameters of the machine learning model and with the chosen dataset. This means collecting these over a series of runs, as input to a single plot.

The code should also give the FPR, FNR, TPR, TNR. In the repository version, these are only identified for the first 100 test images. I need to upload a version that does it for all images.

Per-image quantities

For each image, we get the probability that it belongs to each class ("prediction[]" in the code). FPR etc are calculated based on these, but there is no distinction (I think) between a probability of 0.55 and 0.95. We would like to know, for example, if the correct classification is "protein" what is the distribution of probabilities that these images belong to the protein class (prediction[1]). For those where prediction[1] > 0.5 it is TP, while if prediction[1] < 0.5 it is FN. But we also expect a spread for the TP, if it is not well determined.

Relating back to the original 3D map

I have some code that does this. Must check it in ... Where in the 3D structure are the good predictions, and where the bad? I expect that the images which are hard to classify come from the edge of the protein.