The problem of transforming a set of features (m) to create a new feature set (n) while retaining as much information as possible.

We can use 2 kinds of algorithms:

• Principal Components Analysis (pca)
• Independent Components Analysis (ica)

We commonly use many Measurable features to evaluate a few Latent features.

PCA:

• Minimizing correlation by maximizing variance
• Cares about orthogonality
• Means that a relatively small set of primitive constructs can be combined in a relatively small number of ways to build the control and data structures of the language
• Finding Latent features driving the pattersn in data
• Dimensionality reduction
• Visualize high-dimensional data. It is easier to draw scatterplots with 2-dimensional data
• Reduce noise by throwing away less useful components
• Makes other algorithms work better with fewer inputs : Very high dimensionality might result in overfitting or take up a lot of computing power (time)

ICA:

• Finding independence by converting (through a linear transformation) your input features into a new feature space such that: New features are independent of one another

Property	PCA	ICA
Mutually Orthoganal	✓
Mutually Independent		✓
Maximal Variance	✓
Maximal Mutual Information		✓
Ordered Features	✓
Bag of Features	✓	✓
Blind Source Separation Problem		✓
Directional		✓

• https://www.ritchieng.com/machine-learning-dimensionality-reduction-feature-transform/