Remember, we are in the middle of our theoretical module. It's a lot of definitions, theorems, and proofs. We are building a foundation to get to the applications of linear algebra. Much of what makes linear algebra useful is based on the ideas of linear independence, the span of vectors, and on various subspaces.

We've seen a few important subspaces of $\large \mathbb{R}^n$ - the span of a set of vectors, and the null and column spaces of a given matrix. There are two important concepts that allow us to more precisely describe and use a subspace, and we'll look at those ideas today.

A basis of a subspace is a collection of vectors that allows us to fully understand the space. The definition:

Definition: A set of vectors $\large \lbrace \vec{v_1},\vec{v_2},\dots,\vec{v_m}\rbrace$ in a subspace $\large H$ is called a basis of $\large H$ if:

1. $\large \text{Span}\lbrace\vec{v_1},\vec{v_2},\dots,\vec{v_m}\rbrace=H$
2. $\large \lbrace\vec{v_1},\vec{v_2},\dots,\vec{v_m}\rbrace$ is a linearly independent set of vectors.

As a quick example, the standard unit basis vectors $\large \hat{i}, \hat{j}, \hat{k}$ are indeed a basis for $\large \mathbb{R}^3$. They do span $\large \mathbb{R}^3$ (any vector can be written in terms of $\large \hat{i}, \hat{j}, \hat{k}$, and they do make up a linearly independent set). This example also illustrates the value of a basis. A basis of $\large H$ allows us to write any vector in $\large H$ in terms of the basis vectors, just like we can write any vector in $\large \mathbb{R}^3$ as $\large a\hat{i}+b\hat{j}+c\hat{k}$.

Let's look at an example. Is the following set a basis for $\large \mathbb{R}^3$?

$$\large \begin{bmatrix} 3 \\ 0 \\ -6 \\ \end{bmatrix},\begin{bmatrix} -4 \\ 1 \\ 7 \\ \end{bmatrix},\begin{bmatrix} -2 \\ 1 \\ 5 \\ \end{bmatrix} $$

The solution is worked out in the following video:

If we take the approach of the video on the following sets, we can determine which are bases of $\large \mathbb{R}^3$:

$$\large \begin{bmatrix} 1 \\ 0 \\ 0 \\ \end{bmatrix},\begin{bmatrix} 2 \\ 3 \\ 0 \\ \end{bmatrix} \quad\quad\quad\quad\begin{bmatrix} 1 \\ 0 \\ 0 \\ \end{bmatrix},\begin{bmatrix} 2 \\ 3 \\ 0 \\ \end{bmatrix},\begin{bmatrix} 4 \\ 5 \\ 6 \\ \end{bmatrix},\begin{bmatrix} 7 \\ 8 \\ 9 \\ \end{bmatrix} \quad\quad\quad\quad\begin{bmatrix} 1 \\ 0 \\ 0 \\ \end{bmatrix},\begin{bmatrix} 2 \\ 3 \\ 0 \\ \end{bmatrix},\begin{bmatrix} 4 \\ 5 \\ 6 \\ \end{bmatrix} $$

• The set on the left is not a basis. The vectors are linearly independent, but they don't span $\large \mathbb{R}^3$! (It's too small to be a basis.)
• The set in the middle is not a basis. The vectors span $\large \mathbb{R}^3$, but they aren't linearly independent! (It's too big to be a basis.)
• The set on the right is a basis. It's just right! The vectors are linearly independent and they span $\large \mathbb{R}^3$.

Here's an example where the subspace is not all of $\large \mathbb{R}^n$. Find a basis for the column space of $\large A$:

$$\large A=\begin{bmatrix} 1 & 4 & 0 & 2 & -1 \\ 3 & 12 & 1 & 5 & 5 \\ 2 & 8 & 1 & 3 & 2 \\ 5 & 20 & 2 & 8 & 8 \\ \end{bmatrix} $$

First of all, we know $\large \text{Col}(A)$ is a subspace of $\large \mathbb{R}^4$ since $\large A$ is $\large 4\times 5$ matrix. We also know $\large \text{Col}(A)$ is the span of the column vectors of $\large A$. So maybe the column vectors form a basis for $\large \text{Col}(A)$. How can we see that this is not the case without doing any work?

Click to see the answer

 

Five vectors in $\large \mathbb{R}^4$ cannot be linearly independent! This set is too big. We need to trim it.

 

So how do we find the basis? Let's see in the video:

The video makes use of the following fact: the columns of $\large A$ that end up with leading ones in rref form a basis for $\large \text{Col}(A)$.

Remember, any set $\large B$ of vectors in a subspace $\large H$ is either too big, too little, or just right to be a basis:

• If $\large B$ spans $\large H$ but is not a basis, we can get a basis by removing appropriate vectors from $\large B$.
• If $\large B$ is linearly independent but is not a basis, we can get a basis by adding appropriate vectors to $\large B$.

All bases of $\large H$ will contain the same number of vectors, which leads to our second big definition today:

Definition: The dimension of a vector space $\large V$ (or subspace) is the number of vectors in a basis for $\large V$. We write $\large \text{Dim}(V)$.

Let's see an example. Let's define a subspace of $\large \mathbb{R}^4$. Let $\large S$ be the span of the following three vectors:

$$\large \begin{bmatrix} -1 \\ 2\\ 5\\ 0\\ \end{bmatrix},\begin{bmatrix} 3 \\ 0\\ 1\\ -2\\ \end{bmatrix},\begin{bmatrix} -5 \\ 4\\ 9\\ 2\\ \end{bmatrix} $$

What is $\large \text{Dim}(S)$? The video shows us how to approach this problem.

If we already happen to know the dimension of a vector space, it is easier to tell if a set of vectors is a basis (we only need to check either of the two conditions in the definition of a basis):

Theorem: If $\large \text{Dim}(V)=n$ and if $\large S=\lbrace \vec{v_1}, \vec{v_2}, \vec{v_n}\rbrace$, then:

• If $\large S$ is linearly independent, then $\large S$ is a basis.
• If $\large S$ spans $\large V$, then $\large S$ is a basis.

We have two subspaces associated with a matrix $\large A$. If $\large A$ is $\large m\times n$, then

• $\large \text{Col}(A)$ is a subspace of $\large \mathbb{R}^m$
• $\large \text{Null}(A)$ is a subspace of $\large \mathbb{R}^n$

We give a name to the dimension of each of these subspaces:

Definition: The dimension of $\large \text{Col}(A)$ is called the rank of $\large A$. The dimension of $\large \text{Null}(A)$ is called the nullity of $\large A$.

To find the rank and/or nullity of a matrix, we do the following:

• $\large \text{Rank}(A)=$ number of leading ones in rref
• $\large \text{Nullity}(A)=$ number of free variables in the solution of $\large A\vec{x}=\vec{0}$

Example: find the rank and nullity of A:

$$\large A=\begin{bmatrix} -3 & 6 & -1 & 1 & -7 \\ 1 & -2 & 2 & 3 & -1 \\ 2 & -4 & 5 & 8 & -4 \\ \end{bmatrix} $$

The solution is in the video:

We previously learned that the number of free variables is equal to the number of columns of $\large A$ minus the number of leading ones in the rref. This fact can now be written as a theorem involving rank and nullity:

Theorem: $\large \text{Rank}(A)+\text{Nullity}(A)=$ number of columns of $\large A$.

At this point, we've done all kinds of things with a matrix $\large A$ and a system $\large A\vec{x}=\vec{b}$. Many of those things have overlapped. We can now complete our list of equivalencies: the following are all equivalent facts about a square $\large n\times n$ matrix $\large A$.

1. $\large A$ is invertible.
2. $\large \text{det}(A)\ne 0$.
3. $\large A$ is row equivalent to $\large I$, the identity matrix.
4. $\large A\vec{x}=\vec{0}$ has only the trivial solution.
5. $\large A\vec{x}=\vec{b}$ has a unique solution for any $\large \vec{b}$.
6. $\large \text{Col}(A)=\mathbb{R}^n$.
7. $\large \text{Null}(A)=\lbrace \vec{0}\rbrace$ (the set containing only the zero vector).
8. The columns or rows of $\large A$ are linearly independent.
9. The columns of $\large A$ form a basis for $\large \mathbb{R}^n$.
10. $\large \text{Rank}(A) = n$.
11. $\large \text{Nullity}(A) = 0$.

Definition: A square matrix $\large A$ that meets the above criteria is called nonsingular. Otherwise, the matrix is called singular.

I can:

• Recall and use the definitions of basis, dimension, rank, and nullity.
• Determine if a set of vectors is a basis for a given subspace.
• Find the dimension of a vector space or subspace.
• Find the rank and nullity of a given matrix.