LagrangeMultipliers - crowlogic/arb4j GitHub Wiki

The intuition behind Lagrange multipliers is that at the optimal point, the gradient of the objective function will be in the same direction as the gradient of the constraint function. This is why the method works well in continuous domains where gradients can be easily computed.

For both the system of linear equations and the continuous optimization problem, the key idea is to add a term to your objective function which penalizes violations of your constraints, and the Lagrange multipliers are the weights for these penalty terms. By setting the derivative of this new function equal to zero, you can solve for the values that optimize the original function while satisfying the constraints.