Learning rate - AshokBhat/ml GitHub Wiki

Description

• A [hyperparameter]] that controls the rate of [learning

Mechanism

• During each iteration, the gradient descent algorithm multiplies the learning rate by the gradient
• The resulting product is called the gradient step

FAQ

• When is the learning rate used?
  • During training via gradient descent
• What happens during the gradient descent?
  • During each iteration, the training algorithm multiplies gradient by the learning rate producing a gradient step

• Who decides the learning rate?
  • Decided by the trainer
  • It is a hyperparameter

See also

• Hyperparameter
• [Gradient Descent]] ](/AshokBhat/ml/wiki/[[Gradient-step)
• [Training]] ](/AshokBhat/ml/wiki/[[Model)