Plain network and Residual network - rugbyprof/5443-Data-Mining GitHub Wiki

Plain Network and Residual network

The plain networks are adopted from the VGG nets.The convolutional layers have mostly 3x3 filters and the design follows two rules:

1. For the same output feature map size, the layers have the same number of filters, and
2. If the feature map size is halved, the number of filters is doubled in order to preserve the time complexity per layer.

To convert the plain model to the residual version, shortcut connections are added. The solid line shortcuts are identity mapping. When the dimensions increases there are 2 options (dotted line shortcut):

1. The shortcut still performs identity mapping with zero padding to increasing the dimensions or
2. The shortcut is used to match dimensions utilizing 1x1 convolution.

(https://wiki.tum.de/display/lfdv/Deep+Residual+Networks?focusedCommentId=25007202#comment-25007202)