The primary vector space in this course is $\large \mathbb{R}^n$, but we do want to take a day to point out that the idea extends to other settings. And one of those settings is actually useful! When we introduced vector spaces, this is what we said:

...by carefully listing out what makes a vector a vector, we can see if other things work like vectors. If other things work like vectors, we can use all we know about linear algebra - distance, orthogonality, systems of equations, optimization, geometry, etc. - on those things!

As a reminder, here's the definition of vector space:

A vector space is a set of objects (called vectors) on which the operations of vector addition and scalar multiplication are defined. The following 10 properties (the first 5 deal with vector addition, and the second 5 deal with scalar multiplication) hold:

1. $\large V$ is closed under addition. In other words, the sum of two vectors in $\large V$ is a vector in $\large V$.
2. Vector addition is commutative: $\large \vec{v}+\vec{u}=\vec{u}+\vec{v}$.
3. Vector addition is associative: $\large \vec{u}+\left(\vec{v}+\vec{w}\right) = \left(\vec{u}+\vec{v}\right)+\vec{w}$.
4. $\large V$ has an additive identity, a vector $\large \vec{0}$ such that $\large \vec{v}+\vec{0}=\vec{v}$.
5. Every vector $\large \vec{v}$ in $\large V$ has an additive inverse, a vector $\large -\vec{v}$ such that $\large \vec{v}+(-\vec{v})=\vec{0}$.
6. $\large V$ is closed under scalar multiplication. In other words, $\large c\vec{v}$ is in $\large V$ for any scalar $\large c$ and any vector $\large \vec{v}$.
7. Scalar multiplication is distributive over vector addition: $\large c\left(\vec{u}+\vec{v}\right)=c\vec{u}+c\vec{v}$.
8. Scalar multiplication is distributive over scalar addition: $\large \left(c+d\right)\vec{v}=c\vec{v}+d\vec{v}$.
9. Scalar multiplication is associative: $\large c\left(d\vec{v}\right)=(cd)\vec{v}$.
10. There is a scalar identity 1 such that $\large 1(\vec{v})=\vec{v}$.

Let's look at some other things that meet the criteria to be a vector space. We'll start with some warm up examples to gain familiarity with the definition, and then we'll introduce that actually useful example of a vector space.

Let $\large M_{2,3}$ be the set of all $\large 2 \times 3$ matrices. Addition of matrices is defined, as is scalar multiplication. Do the 10 properties hold? Let's look at one of them. If $\large A$ and $\large B$ are both $\large 2 \times 3$ matrices, is $\large A+B$ also in $\large M_{2,3}$? Yes, so this set is closed under addition. The other 9 properties all hold, so this is a vector space.

What is the dimension of $\large M_{2,3}$? The following matrices make up a basis:

$$\large \begin{bmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \\ \end{bmatrix} \quad \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 0 \\ \end{bmatrix} \quad \begin{bmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \\ \end{bmatrix} \quad \begin{bmatrix} 0 & 0 & 0 \\ 1 & 0 & 0 \\ \end{bmatrix} \quad \begin{bmatrix} 0 & 0 & 0 \\ 0 & 1 & 0 \\ \end{bmatrix}\quad \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} $$

These are linearly independent (none of them can be written as a linear combination of the others), and they span $\large M_{2,3}$. There are 6 vectors in the basis (yes, we are calling the matrices in $\large M_{2,3}$ "vectors"), so $\large \text{Dim}(M_{2,3})=6$.

Let $\large P_2$ be the set of all polynomials of degree 2 or less. Is this set closed under addition? Yes, the sum of two polynomials of degree 2 or less is also a polynomial of degree 2 or less. Is it closed under scalar multiplication? Yes, a constant times a polynomial of degree 2 or less is also a polynomial of degree 2 or less. The other 8 properties are also satisfied, and we have a vector space. Now our "vectors" are things like $p(x) = 3x^2-4x+7$.

What is the dimension of this vector space? What does a basis look like? Consider the functions:

$$\large p_0(x) = 1 \quad\quad p_1(x) = x \quad\quad p_2(x) = x^2$$

Is it possible to write any of these as a linear combination of the others? No, so they are linearly independent. Can we reach any polynomial of degree 2 or less via a linear combination of these? Yes! This set is a basis, and $\large \text{Dim}(P_2)$=3.

Question: We defined $\large P_2$ as the set of polynomials of degree 2 or less. What if we drop the "or less"? Why do we no longer have a vector space?

Click to see the answer

 

Without the "or less," the set is no longer closed under addition. We could have $\large p=2x^2-x+5$ and $\large q=-2x^2+7x+3$. Then $\large p+q = 6x+8$, which is not a polynomial of degree 2.

 

Let's take the vector space $\large M_{2,2}$. Let $\large H$ be the subset of all $\large 2\times 2$ symmetric matrices. Is this subset a subspace? Think about what's required, and make a guess. Then click to see the answer.

Click to see the answer

 

Yes, the zero matrix is part of $\large H$ since it is symmetric. And $\large H$ is closed under addition (the sum of two symmetric $\large 2\times 2$ matrices is a symmetric $\large 2\times 2$ matrix), and it's closed under scalar multiplication. This subset is a subspace (why else would we list it under the category "A subspace of $\large M_{2,2}$"?).

 

Now, the truly useful application of this generalization of vectors is in function spaces. Consider the set of all continuous functions on the interval $\large [a,b]$. Is this a vector space? Well, the sum of two continuous functions is continuous. A continuous function times a scalar is still a continuous function. The other 8 properties all hold! So, yes, this is a vector space, and the functions are the vectors!

Here's an interesting hierarchy of function subspaces:

• Let $\large F$ be the vector space of all functions on $\large [a,b]$.
• Let $\large H_1$ be the set of all integrable functions on $\large [a,b]$. Then $\large H_1$ is a subspace of $\large F$.
• Let $\large H_2$ be the set of all continuous functions on $\large [a,b]$. Then $\large H_2$ is a subspace of $\large F$ and $\large H_1$.
• Let $\large H_3$ be the set of all differentiable functions on $\large [a,b]$. Then $\large H_3$ is a subspace of $\large F$, $\large H_1$, and $\large H_2$.
• Let $\large H_4$ be the set of all polynomials on $\large [a,b]$. Then $\large H_4$ is a subspace of $\large F$, $\large H_1$, $\large H_2$, and $\large H_3$.

The idea of function spaces opens up many STEM applications, especially in data science and differential equations (so, all of hydrology, aerospace, and electrical engineering). There is one function space that is particularly useful and well-known:

Definition: The vector space of $\large L_2$ functions is the set of all functions $\large f$ on $\large [0,2\pi]$ such that $\large \displaystyle \int_{0}^{2\pi} \left|f(x)\right|^2 dx<\infty$. This set is also called the set of square integrable functions.

If these functions spaces are vector spaces, what is their dimension? What does a basis look like? Remember, any vector in the vector space can be written as a linear combination of the basis vectors.

For the $\large L_2$ vector space described above, there is no finite basis, but the following functions turn out to be an infinite basis:

$$\large 1, \sin(x), \cos(x), \sin(2x), \cos(2x), \dots$$

The sine and cosine functions of these integer frequencies are linearly independent, and, miraculously, they do span the function space. What does it mean for them to span the vector space? It means any function can be written as a linear combination of these basis vectors:

$$\large f(x) = \frac{a_o}{2}+\sum_{k=1}^{\infty}a_k\cos(kx)+b_k\sin(kx)$$

This crazy idea - that any function can be written as a linear combination of sines and cosines - is due to Joseph Fourier and is known as a Fourier series (remember from Calc II: a series is a sum of an infinite number of terms, which we certainly have here).

These are big and useful ideas! Fourier series are at the heart of all signal processing and are also very useful in dealing with partial differential equations.

We are presenting this here simply as an example of the usefulness of generalizing vector spaces beyond just $\large \mathbb{R}^n$. The study of function spaces like $\large L_2$ is known as Functional Analysis, and there are multiple courses on this campus that deal with this topic.

I can:

• Recognize if a set is a vector space.
• Find a basis (and the dimension) of simple vector spaces that are not $\large \mathbb{R}^n$.