Lecture 5 - AsyDynamics/CS231n GitHub Wiki

Convolutional neural network

• Similar to the neural network, made up of neurons that have learnable weights and bias, and receive inputs then perform dot product

Architecture overview

• regular neural networks don't scale well to full images, the fully-connected layer require too many neurons
• CNN, use 3d volumes of neuros.

Other notes

• zero padding
• stride
• depth slice
• depth column
• local parameter

Layers used to build CNN

Composed of convolutional layer, pooling layer and fully connected layer

Convolutional layer

Core building block of CNN that does most of the computional heavy lifting.

• local connectivity - dealing with high dimensional input, connect each neuron to local region
• spatial arrangement - depth, stride, zero-padding
• parameter sharing - weights and bias of different region in single slice remain the same; depth slice
• implementation as matrix multiplication
• backpropagation

Pooling layer

common to insert pooling layer in-between Conv layers - reduce spatial size of the representation, reduce amount of parameters and computation

• max pooling, average pooling, L2-norm pooling

Normalization layer

Fully connected layer

difference with CNN, the neurons connect not only local region.

ConvNet Architecture

CONV, POOL, FC

• common pattern: INPUT->[[CONV->RELU]*N->POOL]*M->[FC->RELU]*K->FC