RMSProp - rugbyprof/5443-Data-Mining GitHub Wiki

(Root Mean Square Propagation) is also a method in which the learning rate is adapted for each of the parameters. The idea is to divide the learning rate for a weight by a running average of the magnitudes of recent gradients for that weight. So, first the running average is calculated in terms of means square. https://en.wikipedia.org/wiki/Stochastic_gradient_descent#RMSProp