We've seen that matrices act on vectors with some combination of reflection, shear, and rotation. We think of $\large A$ as a way to transform vectors. For any matrix $\large A$, however, there is a certain direction that is impervious to the transformation! These directions, different for every $\large A$, end up being quite useful in differential equations and other engineering applications.

Let's look at a few specific matrices:

• A reflection matrix:

$$\large A=\begin{bmatrix} -1 & 0 \\ 0 & 1\\ \end{bmatrix} $$

This matrix reflects vectors. For example, we have

$$\large A\begin{bmatrix} 2\\ 3\\ \end{bmatrix} = \begin{bmatrix} -2\\ 3\\ \end{bmatrix}, \text{ and } A\begin{bmatrix} -5\\ -1\\ \end{bmatrix} = \begin{bmatrix} 5\\ -1\\ \end{bmatrix} $$

But notice that it has no effect on vectors in a certain direction:

$$\large A\begin{bmatrix} 0\\ 1\\ \end{bmatrix} = \begin{bmatrix} 0\\ 1\\ \end{bmatrix}, \text{ and } A\begin{bmatrix} 0\\ -2\\ \end{bmatrix} = \begin{bmatrix} 0\\ -2\\ \end{bmatrix} $$

It seems anything along the $\large y$ axis doesn't get transformed by $\large A$. Here we see the action of $\large A$ on $\large \left<2,3\right>$ and on $\large \left<0,1\right>$:

• Now consider a shear matrix:

$$\large A=\begin{bmatrix} 1 & 2 \\ 0 & 1\\ \end{bmatrix} $$

This will shear space parallel to the $\large x$ axis. For example,

$$\large A\begin{bmatrix} 2\\ 3\\ \end{bmatrix} = \begin{bmatrix} 7\\ 3\\ \end{bmatrix} $$

But if we apply this matrix to vectors along the $\large x$ axis, they are unchanged:

$$\large A\begin{bmatrix} 4\\ 0\\ \end{bmatrix} = \begin{bmatrix} 4\\ 0\\ \end{bmatrix} $$

Here we see the action of $\large A$ on $\large \left<2,3\right>$ and on $\large \left<4,0\right>$:

• Let's think about a less obvious matrix:

$$\large A=\begin{bmatrix} 2 & -12 \\ 1 & -5\\ \end{bmatrix} $$

Uh...what does this matrix do to vectors? We know it sends $\large \hat{\imath}$ to $\large \left<2,1\right>$ and $\large \hat{\jmath}$ to $\large \left<-12,-5\right>$, so the axes definitely get transformed by $\large A$. But even this matrix has a special direction, as seen here:

$$\large A\begin{bmatrix} 3\\ 1\\ \end{bmatrix} = \begin{bmatrix} -6\\ -2\\ \end{bmatrix}=-2\begin{bmatrix} 3\\ 1\\ \end{bmatrix}, \text{ and } A\begin{bmatrix} -9\\ -3\\ \end{bmatrix} = \begin{bmatrix} 18\\ 6\\ \end{bmatrix}= -2\begin{bmatrix} 3\\ 1\\ \end{bmatrix} $$

It seems anything in the $\large \left<3,1\right>$ direction just gets scaled by $\large A$ but stays along that same line. Here we see the vector $\large \left<3,1\right>$ and the action of $\large A$ on both $\large\left<3,1\right>$ (blue) and on $\large \left<5,1\right>$ (green). We see that the blue vector is along the same line as $\large \left<3,1\right>$, but the green vector is not:

We're going to give a name to the special direction that only gets scaled by a matrix:

Definition: If $\large A$ is an $\large n\times n$ matrix, a scalar $\large \lambda$ is an eigenvalue of $\large A$ if there is a nonzero vector $\large \vec{v}$ such that

$$\large A\vec{v}=\lambda\vec{v}$$

The vector $\large \vec{v}$ in this case is called an eigenvector of $\large A$.

We can easily check if a given vector is an eigenvector. The video will show that $\large \vec{v}=\left<2,-3,-1\right>$ is an eigenvector of $\large A$:

$$\large A=\begin{bmatrix} -2 & -4 & 2\\ -2 & 1 & 2 \\ 4 & 2 & 5 \\ \end{bmatrix} $$

Turns out, this special direction is useful in many places.

• In differential equations, eigenstuff is necessary to solve systems of equations. The signs of eigenvalues also indicate stability of solutions.
• In data science, eigenstuff is used to identify important dimensions of datasets, which allows us to reduce data problems to manageable size (we will see some of this when we discuss the SVD later in this class).
• Eigenvectors are at the heart of PageRank, an algorithm used to rank results of an internet search to give the most relevant results first.
• In structural engineering, eigenstuff is used to find the critical levels under which materials will buckle.
• Many Mines students will take a Finite Element Analysis class (required for MechE's). This technique relies heavily on eigenstuff.

Given that these are important quantities, how do we find the eigenvectors and eigenvalues for $\large A$? We call this solving the eigenproblem. So, how do we solve an eigenproblem for a given $\large A$? Let's start with the definition:

$$\large \begin{split} &A\vec{v}=\lambda\vec{v}\\ &\Rightarrow A\vec{v}-\lambda\vec{v}=\vec{0}\\ &\Rightarrow \left(A-\lambda I\right)\vec{v}=\vec{0} \end{split} $$

This last equation is a system of linear equations. It certainly has the trivial solution $\large \vec{v}=\vec{0}$, but eigenvectors must be nonzero. When will a system $\large \left(A-\lambda I\right)\vec{v}=\vec{0}$ also have a nonzero solution (when will it have infinitely many solutions)? Only when $\large \left(A-\lambda I\right)$ is singular! When is $\large \left(A-\lambda I\right)$ singular? When $\large \text{det}\left(A-\lambda I\right)=0$. So, to find the eigenvalues of $\large A$, we need to find values of $\large \lambda$ where:

$$\large \text{det}\left(A-\lambda I\right)=0$$

Definition: The left hand side of the equation above is a polynomial in $\large \lambda$, and we call it the characteristic polynomial. The roots of the characteristic polynomial are the eigenvalues of $\large A$.

Once we have the eigenvalues from the equation above, we can go back to the definition to get the eigenvectors:

$$\large \left(A-\lambda I\right)\vec{v}=\vec{0}$$

This is a system of equations that we solve with Gaussian elimination.

Let's see how this is done on an actual example. Find the eigenvalues and eigenvectors for $\large A$:

$$\large A= \begin{bmatrix} 2 & -12\\ 1 & -5 \\ \end{bmatrix} $$

The solution is in the video:

We see the steps for an eigenproblem:

1. Find the characteristic polynomial $\large \text{det}\left(A-\lambda I\right)$
2. Find the roots of the polynomial (either factor it, or use the quadratic equation if it's a quadratic).
3. For each $\large \lambda$, solve $\large \left(A-\lambda I\right)\vec{v}=\vec{0}$ for $\large \vec{v}$.

Let's see another example. Solve the eigenproblem for $\large A$:

$$\large A= \begin{bmatrix} 1 & 0 &-1\\ 1 & 2 & 0\\ 4 & 0 & 5\\ \end{bmatrix} $$

Here's the solution:

We started this section looking for vectors that are only scaled by $\large A$. What about a rotation matrix? Every nonzero vector gets rotated by a rotation matrix:

Does that mean rotation matrices have no eigenvectors? Well, let's try. For a simple rotation,

$$\large A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \\ \end{bmatrix} $$

We have $\large \text{det}\left(A-\lambda I\right)=\lambda^2+1$. This characteristic polynomial has no real roots, but it does have complex roots, $\large \lambda = \pm i$.

Let's see how this works in a full example. Solve the eigenproblem for $\large A$:

$$\large A=\begin{bmatrix} 1 & -2\\ 1 & 3\\ \end{bmatrix} $$

The solution is in the video:

We've been looking at small matrices, but it's often large matrices we deal with in applications. In a lab, we will see how we can have Python do this for us. In the meantime, here are some facts to know:

• If $\large A$ is $\large n\times n$, then the characteristic polynomial is an $\large n^{th}$ degree polynomial, so there are $\large n$ eigenvalues (including repeated eigenvalues). If an eigenvalue is repeated $\large k$ times, we say it has multiplicity $\large k$.
• All complex eigenvalues and eigenvectors come in complex conjugate pairs. That is, if $\large \lambda = 3+2i$ is an eigenvalue, then so is $\large \lambda = 3-2i$.
• If $\large A$ is diagonal or triangular, the eigenvalues are just the elements along the diagonal of $\large A$.
• If $\large \lambda=0$ is an eigenvalue, then $\large \text{det}(A)=0$, so $\large A$ is singular.
• If an $\large n\times n$ matrix $\large A$ has $\large n$ distinct eigenvectors, then they are linearly independent (and therefore form a basis for $\large \mathbb{R}^n$).
• For a repeated eigenvalue of multiplicity $\large k$, there may be anywhere from 1 through $\large k$ linearly independent eigenvectors. The number of linearly independent eigenvectors is the dimension of the nullspace of $\large \left(A-\lambda I\right)$ (the nullity, which is also number of free variables in the solution).

Let's throw in a new term as well:

The last bullet point above is worth expanding a bit. The eigenvectors of $A$ are the solutions of $\large \left(A-\lambda I\right)\vec{v}=\vec{0}$. We know there will be infinitely many solutions. We've seen examples where these all lie along a single line (where is one free variable in the solution).

But the solution space can be larger (there can be more than one free variable in the solution). We call the solution space the eigenspace:

Defnition: The eigenspace for $\lambda$ is the null space of $\large \left(A-\lambda I\right)$.

Let's say a $\large 7\times 7$ matrix $\large A$ has an eigenvalue $\large \lambda=4$ repeated with multiplicity 3. Here's what can happen with the eigenvectors:

• If $\large \left(A-3I\right)$ has rank 6, then it has nullity 1, and there is one free variable in the solution of the eigenvector equation. There is only one linearly independent eigenvector for $\large \lambda=4$.
• If $\large \left(A-3I\right)$ has rank 5, then it has nullity 2, and there are two free variables. There are two linearly independent eigenvectors for $\large \lambda=4$.
• If $\large \left(A-3I\right)$ has rank 4, then it has nullity 3, and there are three free variables. There are three linearly independent eigenvectors for $\large \lambda=4$.

I can:

• Determine if a given vector is an eigenvector of a given matrix.
• Find the eigenvalues and eigenvectors of $\large A$ (by hand for $\large 2\times 2$ matrices, including real and complex eigenvalues).
• Find the eigenspace for a given $\large A$ and $\large \lambda$.