learning rate - AshokBhat/ml GitHub Wiki

• A hyperparameter that controls the rate of learning

• During each iteration, the gradient descent algorithm multiplies the learning rate by the gradient
• The resulting product is called the gradient step

• When is the learning rate used?
  • During training via gradient descent
• What happens during the gradient descent?
  • During each iteration, the training algorithm multiplies gradient by the learning rate producing a gradient step

• Who decides the learning rate?
  • Decided by the trainer
  • It is a hyperparameter

• Hyperparameter
• Gradient Descent | Gradient step
• Training | Model