Learning rate decay - AshokBhat/ml GitHub Wiki About A hyperparameter Mechanism First, adopt an initially large learning rate Then, decay it by a certain factor after pre-defined epochs. See also Learning rate Hyperparameter Epoch