In an earlier section, we read "Essentially, the rank of a matrix is an indicator of how much of the information in the matrix is unique and how much is redundant." We are now in a position to make the idea of "redundant" rows in a matrix more explicit.

A set of vectors is called linearly independent if the set contains no redundant information. Mathematically, this leads to the following equivalent definitions:

Definition: A set of vectors $\large \vec{v_1},\vec{v_2},\dots,\vec{v_m}$ is linearly independent if none of the vectors can be written as a linear combination of the others. The set is linearly dependent if at least one of the vectors can be written as a linear combination of the others.

Linear independence is how we measure the amount of redundancy in a dataset or a system of equations. It's how we quantify the amount of "unique" information in a matrix.

With our new definition of linearly independent, we can update our definition of the rank of a matrix.

Old definition: The rank of a matrix $\large A$ is the number of leading ones in rref of $\large A$.

New definition: The rank of a matrix $\large A$ is the number of linearly independent rows (or columns, it turns out this number is always the same for all matrices) of $\large A$.

The definitions above are the most intuitive definitions for linear independence/dependence, but there is an alternate, equivalent definition that is often useful in proving things:

Definition: A set of vectors $\large \vec{v_1},\vec{v_2},\dots,\vec{v_m}$ is linearly independent if the system of linear equations:

$$\large c_1\vec{v_1}+c_2\vec{v_2}+\dots+c_m\vec{v_m}=0$$

has only one solution, $\large c_1=c_2=\dots=c_m=0$, known as the trivial solution. Otherwise, it is linearly dependent.

Let's see how this is an equivalent definition:

Consider the system of equations in this definition: $\large c_1\vec{v_1}+c_2\vec{v_2}+\dots+c_m\vec{v_m}=0$. Let's say there's a solution with $\large c_1\ne 0$. Then we can write $\large c_1\vec{v_1}=-c_2\vec{v_2}-\dots-c_m\vec{v_m}$, so we can write $\vec{v_1}$ as a linear combination of the others, and the set of vectors is linearly dependent.

Just some facts about linear independence:

1. The columns of a matrix $\large A$ are linearly independent if $\large A\vec{x}=\vec{0}$ has only the trivial solution ($\large x_1=x_2=\dots=x_n=0$).
2. Two vectors are linearly dependent if one is a constant, nonzero multiple of the other.
3. A set of vectors $\large \lbrace\vec{v_1}, \vec{v_2}, \dots, \vec{v_p}\rbrace$ in $\large \mathbb{R}^n$ is linearly dependent if $\large p>n$. So, four vectors in $\large \mathbb{R}^3$ necessarily make up a linearly dependent set.
4. If $\large \vec{0}$ is in the set, the set is linearly dependent.

To see if a given set of vectors is linearly independent, we set up the linear system from the definition, and we solve with Gaussian elimination.

As an example: are the vectors $\large \vec{v_1}=\left< 1,2,3\right>$, $\large \vec{v_2}=\left<4,5,6\right>$, and $\large \vec{v_3}=\left<2,1,0\right>$ linearly independent? Let's see how to do this in the video:

Why does this definition matter? We will see much more in the next lecture, but for now, we can state a few facts with geometry:

Each image shows two vectors in $\large \mathbb{R}^2$:

• In the image on the left, the two vectors are linearly dependent (one is just 2 times the other), and the span of the two vectors is just a line.
• In the image on the right, the two vectors are linearly independent, and the span of the two vectors is all of $\large \mathbb{R}^2$, meaning we can write any vector in $\large \mathbb{R}^2$ as a linear combination of these two vectors.
• We see that a set of linearly independent vectors has more "reach" than a set of dependent vectors.

With linear independence, we can now add to our list of equivalencies for square matrices:

The following are all equivalent facts about a square $\large n\times n$ matrix $\large A$.

1. $\large A$ is invertible.
2. $\large \text{det}(A)\ne 0$.
3. $\large A$ is row equivalent to $\large I$, the identity matrix.
4. $\large A\vec{x}=\vec{0}$ has only the trivial solution.
5. $\large A\vec{x}=\vec{b}$ has a unique solution for any $\large \vec{b}$.
6. $\large \text{Col}(A)=\mathbb{R}^n$.
7. $\large \text{Null}(A)=\lbrace \vec{0}\rbrace$ (the set containing only the zero vector).
8. The columns or rows of $\large A$ are linearly independent.

Definition: A square matrix $\large A$ that meets the above criteria is called nonsingular. Otherwise, the matrix is called singular.

After this this section, I can:

• Recall and use the definitions of linear independence and linear dependence.
• Determine if a set of vectors is linearly independent.