Linear Algebra deals primarily with matrices, which are boxes of numbers. Here's an example of a matrix:

$$ \large A = \begin{bmatrix} 3 & -1 & 4\\ -5 & 0 & 2\\ \end{bmatrix} $$

We typically use capital letters to denote matrices. The matrix $\large A$ in the example above is a $\large 2\times 3$ matrix, which means it has 2 rows and 3 columns. A matrix with $\large m$ rows and $\large n$ columns is an $\large m\times n$ matrix, and we call this the size of the matrix. To pick out an individual element of a matrix (one number in the box), we use a subscript notation. In the example above, we could write $\large A_{1,2}=-1$ because the element in the first row and second column is a $\large -1$.

Now, why do we worry about boxes of numbers? Aside from just the interesting math that arises, these boxes of numbers are usually storing information. The study of matrices gives us power to manipulate and analyze that data.

This is a list of some basic matrix operations with examples:

• Addition/subtraction works by corresponding components:

$$\large \begin{bmatrix} 2 & 3 \\ 4 & -1 \\ \end{bmatrix} + \begin{bmatrix} 1 & 0 \\ -2 & 6 \\ \end{bmatrix} = \begin{bmatrix} 3 & 3\\ 2 & 5 \\ \end{bmatrix}$$

• Scalar multiplication also works by component:

$$\large 3\begin{bmatrix} 6 & -4 \\ 2 & 0 \\ 3 & 1 \\ \end{bmatrix} = \begin{bmatrix} 18 & -12\\ 6 & 0 \\ 9 & 3\\ \end{bmatrix}$$

• The transpose of a matrix, denoted $\large A^T$, is the matrix we get if we turn the rows of $\large A$ into columnns (or, equivalently, the columns into rows):

$$\large \begin{bmatrix} 6 & -4 \\ 2 & 0 \\ 3 & 1 \\ \end{bmatrix}^T = \begin{bmatrix} 6 & 2 & 3\\ -4 & 0 & 1 \\ \end{bmatrix}$$

• A square matrix is a matrix that has the same number of rows and columns (a $\large 2\times 2$, a $\large 3\times 3$, etc.)

• The identity matrix $\large I$ is a square matrix with this form:

$$\large I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} $$

• A zero matrix can be any size and consists only of 0's.

• Vectors can be considered 1-dimensional matrices. So,

$$\large \vec{x} = \begin{bmatrix} 3 \\ -2 \\ 1\\ \end{bmatrix} \text{, a column vector is a } 3\times 1 \text{ matrix } $$

$$\large \vec{y} = \begin{bmatrix} 4 & 1 & -1 & 5\\ \end{bmatrix} \text{, a row vector is a } 1\times 4 \text{ matrix } $$

• The diagonal of a square matrix is the set of numbers connecting the upper left element to the bottom right element. This is sometimes written as a vector and sometimes just as a list of numbers.

The product of two matrices has a trickier definition than addition or subtraction. The product $\large C=AB$ is defined as follows:

$$\large C_{i,j}=\text{ the dot product of the ith row of A with the jth column of B}=\sum_{k=1}^nA_{i,k}B_{k,j}$$

The image below illustrates the definition:

^{Image from: https://www.mathsisfun.com/algebra/matrix-multiplying.html}

Here's a video example of how it works:

Because of this definition, matrix multiplication is not always defined. For $\large C=AB$ to exist, the number of columns of $\large A$ must equal the number of rows of $\large B$. The following image shows the common way to see if $\large AB$ is defined, which also gives us a way to determine the size of $\large AB$. We line up the sizes of $\large A$ and $\large B$, and the inner dimensions must be equal for the the product to be defined. If the product is defined, the outer dimensions give us the size of the product.

Since a vector is just a one dimensional matrix, matrix-vector multiplication is just a small case of matrix-matrix multiplication. In the following example, we have a $\large 2\times 3$ matrix times a $\large 3 \times 1$ column vector. The inner dimensions agree, so the product is defined. The outer dimensions tell us that the product will be a $\large 2\times 1$ vector. Here it is:

$$\large \begin{bmatrix} 3 & 8 & 0\\ -1 & 6 & -2\\ \end{bmatrix} \begin{bmatrix} 4 \\ -3 \\ 1 \\ \end{bmatrix} = \begin{bmatrix} (3)(4)+(8)(-3)+(0)(1) \\ (-1)(4)+(6)(-3)+(-2)(1) \\ \end{bmatrix} = \begin{bmatrix} -12 \\ -24 \\ \end{bmatrix} $$

$$\large \begin{bmatrix} 3 & 8 & 0\\ -1 & 6 & -2\\ \end{bmatrix} \begin{bmatrix} 4 \\ -3 \\ 1 \\ \end{bmatrix} = 4\begin{bmatrix} 3 \\ -1\\ \end{bmatrix}-3\begin{bmatrix} 8 \\ 6\\ \end{bmatrix}+\begin{bmatrix} 0 \\ -2\\ \end{bmatrix} = \begin{bmatrix} -12 \\ -24 \\ \end{bmatrix}$$

Linear algebra provides a slick way of representing systems of linear equations. Take the following system as an example:

$$\large \begin{split} 2x_1+x_2+x_3&=-2\\ -x_1+4x_2-3x_3&=8\\ x_1+x_2-x_3&=1\\ \end{split} $$

Now, let's define three matrices/vectors as follows:

$$\large A = \begin{bmatrix} 2 & 1 & 1 \\ -1 & 4 & -3 \\ 1 & 1 & -1 \\ \end{bmatrix}, \vec{x} = \begin{bmatrix} x_1\\ x_2\\ x_3\\ \end{bmatrix}, \vec{b} = \begin{bmatrix} -2\\ 8\\ 1\\ \end{bmatrix} $$

Then we see that the matrix-vector product equation $\large A\vec{x}=\vec{b}$ is equivalent to the system of equations we started with! This is the justification for the funny definition of matrix multiplication we promised in lecture 2. The matrix $\large A$ is called the coefficient matrix (since it contains the coefficients of the variables in our system), and the vector $\large \vec{b}$ is called the right hand side vector.

Check your understanding: What system of equations is represented by the following matrix equation?

$$\large \begin{bmatrix} 2 & 3 \\ -1 & 4 \\ \end{bmatrix} \vec{x}=\begin{bmatrix} 3\\ 0\\ \end{bmatrix} $$

Click to see the answer

 

The system consists of the equations $\large 2x+3y=3$ and $\large -x+4y=0$.

 

Whenever we see $\large A\vec{x}=\vec{b}$, we should recognize that we are dealing with a system of linear equations. This representation is great for two reasons:

1. It is compact, so we can still write a large system of many equations and unknowns on a single line.
2. It will lead to a systematic approach (an algorithm) to find the solution.

Some matrices come up often enough to get names. The following definitions will appear from time to time in this class:

• A square matrix $\large A$ is symmetric if $\large A^T=A$. Here's an example:

$$\large \begin{bmatrix} 4 & -1 & 0 & 5\\ -1 & 10 & 3 & -3\\ 0 & 3 & -2 & 12\\ 5 & -3 & 12 & 8 \\ \end{bmatrix} $$

• A square matrix $\large A$ is diagonal if $\large a_{i,j}=0$ for all $\large i\ne j$ (every element not on the diagonal is zero). Here's an example:

$$\large \begin{bmatrix} -2 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0\\ 0 & 0 & 6 & 0 & 0\\ 0 & 0 & 0 & -7 & 0\\ 0 & 0 & 0 & 0 & 10\\ \end{bmatrix} $$

• A square matrix $\large A$ is upper (lower) triangular if every element below (above) the diagonal is zero. Here are examples of each:

$$\large \begin{bmatrix} -2 & 3 & 7 & 0 & -1 \\ 0 & 1 & 2 & -5 & 14\\ 0 & 0 & 6 & 4 & -4\\ 0 & 0 & 0 & -7 & 1\\ 0 & 0 & 0 & 0 & 10\\ \end{bmatrix}\quad\quad\quad\quad \begin{bmatrix} 4 & 0 & 0 & 0 & 0\\ -2 & 1 & 0 & 0 & 0\\ 1 & 0 & -3 & 0 & 0\\ 5 & -1 & 2 & 0 & 0\\ 3 & 7 & -10 & 1 & 4\\ \end{bmatrix} $$

• A square matrix $\large A$ is orthogonal if its columns are all orthogonal to each other (the dot product of any column with any other column is 0) and if every column is a unit vector (has length one). An equivalent definition is that $\large A^TA=AA^T=I$, the identity matrix. Here's an example:

$$\large \begin{bmatrix} 1/3 & 2/3 & -2/3 \\ -2/3 & 2/3 & 1/3 \\ 2/3 & 1/3 & 2/3 \\ \end{bmatrix} $$

Every row and column in this matrix has length 1. Go ahead and check the dot product of any row with another. They are all zero.

There are many more special, named types of matrices, but these four will do for now.

I can:

• Use linear algebra terminology and notation.
• Perform basic matrix computations.
• Multiply matrices together (or recognize when the matrix product does not exist).
• Write a system of linear equations as $\large A\vec{x}=\vec{b}$.
• Recognize square, symmetric, diagonal, triangular, and orthogonal matrices.