Stanford Linear Decoder - SummerBigData/MattRepo GitHub Wiki

In this exercise we replaced our sigmoid function that was used to compute the output layer with a linear function in our sparse autoencoder to create a linear decoder. We used it to learn features off of images from the STL10 - dataset. This data set consisted of (96, 96) images where 5000 of them were for training and 8000 were for testing. There were a total of 10 classes: airplane, bird, car, cat, deer, dog, horse, monkey, ship, truck. We were provided with 100k (8, 8) patches of the images from the STL-10 dataset. The images this time were RGB instead of greyscale. When training we needed to subtract the mean from our matrix of patches and whiten them. This is what an image would look like before and after whitening:

We then saved our whitening matrix, mean patches matrix, and our weights to use later in convolution and pooling. We also had to whiten our weights when we were visualizing our data in order for our images of the max activations for each input to display correctly. We used the same whitening matrix that was used on our patches. Here is what the max activations look like:

We also checked the accuracy of out linear decoder by comparing the outputs against the inputs. Here are what some of the images look like side by side:

Our error ended up being 14.9%.