Fully connected layers - rugbyprof/5443-Data-Mining GitHub Wiki

CNNs have one more arrow in their quiver. Fully connected layers take the high-level filtered images and translate them into votes. In our case, we only have to decide between two categories, X and O. Fully connected layers are the primary building block of traditional neural networks. Instead of treating inputs as a two-dimensional array, they are treated as a single list and all treated identically. Every value gets its own vote on whether the current image is an X or and O. However, the process isn’t entirely democratic. Some values are much better than others at knowing when the image is an X, and some are particularly good at knowing when the image is an O. These get larger votes than the others. These votes are expressed as weights, or connection strengths, between each value and each category.

When a new image is presented to the CNN, it percolates through the lower layers until it reaches the fully connected layer at the end. Then an election is held. The answer with the most votes wins and is declared the category of the input.

Fully-Connected layer implies that every neuron in the previous layer is connected to every neuron on the next layer. High-level features of the input image are derived from the outputs of convolutional and pooling layers. Fully Connected layer uses these features for classifying the input image into various classes based on the training dataset. This is the cheap way of learning non-linear combinations of these features.