Regression - shivamvats/notes GitHub Wiki

Note: Likelihood of parameter theta is p(D|theta).

Least Squares and MLE

  • Gaussian Prior on beta: Biases solution towards small beta. Is equivalent to Ridge Regression.
  • Laplace Prior on beta: Biases solution towards small beta. Is equivalent to LASSO.

Under the assumption of zero mean Gaussian noise, the Least Square Estimate is the same as MLE.

Regularized Least Squares and MAP

Ridge Regression

Wiki

LASSO (Least Absolute Shrinkage and Selection Operator)

⚠️ **GitHub.com Fallback** ⚠️