LR: BestFit line - dudycooly/1235 GitHub Wiki

Conceptually, OLS technique tries to reduce the sum of squared errors ∑[Actual(y) - Predicted(y')]² by finding the best possible value of regression coefficients (β0 or b, β1 or m). ie. where

By changing different values of m and b, we can obtain S. Plotting those in graph would get something like this

As you notice, after the point where S is minimum, the slope is zero or reversing its direction. This is our point of best fit line which is represented by optimal values of m and b that we are looking for

To find these points mathematically,

i) we need to find rate of change S with respect to m and b individually The partial derivatives would give them

ii) minimising (In algebraic term, setting to zero) these partial derivatives we would get intercept (b) and slope (m) as follows:

this can be written as

$b=\frac{\sum{y}}{n}-\frac{m\sum{x}}{n}$

$m=\frac{n\sum{xy}-\sum{x}\sum{y}}{n\sum{x^{2}}-(\sum{x})^{2}}$

Now substituting $\frac{\sum{x}}{n}=\bar{X}$ to get rid of sum symbol

$b=\bar{y}-m\bar{x}$

$m=\frac{n.n.\bar{xy}-n\bar{x}n\bar{y}}{n.n.\bar{x^{2}}-(n\bar{x})^{2}}=\frac{\bar{xy}-\bar{x}.\bar{y}}{\bar{x^{2}}-(\bar{x})^{2}}$