Let $X_t$ be a zero-mean, square-integrable stochastic process with autocovariance function $R(s, t)$. The Karhunen-Loève theorem states that there exists an orthonormal set of functions { \phi_k(t) } such that $X_t$ can be represented as an infinite series:

X_t = \sum_{k=1}^{\infty} Z_k \phi_k(t)

Here, the $Z_k$ are uncorrelated random variables defined by

$$Z_k = \int_{-\infty}^{\infty} X_t \phi_k(t) dt$$,

and the functions $\phi_k(t)$ are the eigenfunctions of the autocovariance function. They satisfy the integral equation:

\int_{-\infty}^{\infty} R(s, t) \phi_k(t) dt = \lambda_k \phi_k(s)

where $\lambda_k$ are the eigenvalues of the autocovariance function and are always real and nonnegative.

The eigenfunctions are orthogonal, i.e.,

\int_{-\infty}^{\infty} \phi_k(t) \phi_j(t) dt = \delta_{kj}

where $\delta_{kj}$ is the Kronecker delta function, which equals 1 if $k=j$ and 0 otherwise.

The uncorrelated random variables $Z_k$ have zero mean and variances equal to the corresponding eigenvalues:

E[Z_k^2] = \lambda_k

and

E[Z_k] = 0

Thus, the Karhunen-Loève theorem provides a complete orthonormal set of functions $\phi_k(t)$ that can be used to represent the stochastic process ${X_t}$ as an infinite series of uncorrelated random variables.

See also GaussianProcessOrthogonalPolynomials.

Answering the Sturm-Liouville question

The process of determining the orthogonal eigenfunctions corresponding to a covariance kernel, which defines the inner product space, is generally known as "solving the eigenproblem" or "solving the Sturm-Liouville problem" in the context of functional analysis. In this context, the procedure involves finding the eigenvalues and corresponding eigenfunctions of the operator defined by the covariance kernel.

This process can be described as follows:

1. The covariance kernel defines an integral operator. The application of this operator to a function is given by the integral of the covariance kernel times the function.
2. The eigenfunctions and eigenvalues of this operator are determined by solving the integral equation:

\int_{-\infty}^{\infty} R(s, t) \phi_k(t) dt = \lambda_k \phi_k(s)

3. Here, $R(s,t)$ is the covariance kernel, $\lambda_k$ are the eigenvalues, and $\phi_k(t)$ are the eigenfunctions.

In the specific context of the Karhunen-Loève theorem, the eigenfunctions are also orthonormal with respect to the inner product defined by the covariance kernel, making them suitable for use in expanding the stochastic process in terms of these eigenfunctions.

While there's not a specific name for this process in the context of the Karhunen-Loève theorem, it's sometimes referred to as the "Karhunen-Loève transform" or "Karhunen-Loève decomposition", although these terms also refer to the resulting representation of the stochastic process in terms of the eigenfunctions.