ReproducingKernelHilbertSpace - crowlogic/arb4j GitHub Wiki
The Role of Inner Products and Reproducing Kernel Hilbert Space(RKHS) in Stochastic Processes
Inner Products and RKHS
A Reproducing Kernel Hilbert Space (RKHS) is a particular kind of Hilbert space where functions serve as elements. The inner product in this space can be induced by a kernel function $k(s, t)$. Specifically, for any two functions $f, g \in \mathcal{H}$ within an RKHS $\mathcal{H}$, the inner product is given by:
$$\langle f, g \rangle = \int \int f(s) \overline{g(t)} k(s, t) ds dt$$
This construct ensures that the inner product is a strict inner product in $\mathcal{H}$.
Riesz Representation Theorem
Theorem
For every continuous linear functional $F$ on a Hilbert space $\mathcal{H}$, there exists a unique element $y$ in $\mathcal{H}$ such that:
$$F(x) = \langle x, y \rangle \forall x \in \mathcal{H}$$
Implications for RKHS
In the case of RKHS, the Riesz Representation Theorem ensures that the evaluation functional $F(f) = f(s)$ can be represented as an inner product:
$$F(f) = f(s) = \langle f, k(s, \cdot) \rangle$$
This verifies that the kernel $k(s, \cdot)$ effectively represents function evaluations and functionals in the RKHS, thereby making $k(s, \cdot)$ a reproducing kernel.
Stochastic Processes and Covariance Function
When $f(s) = X(s)$ and $g(s) = X(t)$ for a stochastic process $X \in \mathcal{H}$, the covariance function $C(s, t)$ is naturally represented as:
$$C(s, t) = \langle X(s), X(t) \rangle$$
In this special case, $C(s, t)$ becomes a true inner product in $\mathcal{H}$. Here, $k(s, t)$ serves as the autocovariance kernel function of $X$.