SVD - shivamvats/notes GitHub Wiki

Every matrix A has an SVD decomposition: A = USV'.

In its full form, if A is an m x n matrix, then U is an m x m unitary (orthogonal) matrix, S is an m x n diagonal matrix and V is an n x n unitary (orthogonal) matrix.

Implies A is a full rank n x n matrix.

Ax = b

Then the solution is clearly, x = inv(A)b = V.inv(S).U'.b.

A is m x n where m > n.

• U: m x m
• S: m x n. Only the first r = rank(A) diagonal elements are non-zero.
• V: n x n

No exact solution to this system of equations exists as S cannot be inverted. However, it can be shown that the solution obtained by considering just the first r columns of U, S and V is the Least Square solution.

Reference