History of statistical learning - ricket-sjtu/bi028 GitHub Wiki

Inspired by the advent of machine learning and other disciplines, statistical learning has emerged as the subfield of statistics, focused on the nonsupervised and supervised modeling and predictions. Nowadays, statistical learning has transitted from academic to mainstream field.

• 1900s, Legendre and Gauss

Least squres -> linear regression

• 1936, Fisher

Linear discriminant analysis

• 1940s, various authors

Logistic regression

• 1970s, Nelder and Wedderburn

Generalized linear models (GLMs)

• 1980s, Breiman, Friedman, Olshen and Stone

Classification and regression tree (CART); Cross-validation for model selection

• 1986, Hastie and Tibshrani

Generalized additive models (GAMs), nonlinear extension to GLMs

<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=default"></script>

• \(n)\: number of observations;
• \(p)\: number of variables;
• $\mathbf{X}$: $n \times p$ matrix of observations;
• $x_{ij}$: $j$-th variable for the $i$-th observation;