Rylan Schaeffer

Kernel Papers

Hilbert Space

A Hilbert space \(\mathcal{H}\) is an inner product space that is separable and -complete. Separable means and complete means that the limits of all Cauchy Sequences are also in \(\mathcal{H}\) e.g. \(\mathbb{R}\) is ok, but \(\mathbb{Q}\) is not.

Example Hilbert Spaces


Continuity is necessary! Otherwise, this doesn’t work. For example, suppose we want to find \(f\) to construct the functional \(F_f(g) := g(x)\). We know that we must be able to write this functional as an inner product \(\int g(t) f(t)^* dt\), but the only function that can result in \(g(x) = \int g(t) f(t)^* dt\) is the Dirac delta function \(\delta(x)\), but it doesn’t belong to the Hilbert space!

Bases for Hilbert Spaces

One common orthonormal basis for the Hilbert space \(L^2([0, T])\) is the Fourier basis, defined as:

\[\phi_z(t) := \frac{1}{\sqrt{T}} e^{i \frac{2 \pi z}{T} t}\]

This basis is orthonormal because

\[\begin{align*} \langle \phi_n, \phi_m \rangle_{\mathcal{H}} &= \frac{1}{T} \int_{0}^T e^{i \frac{2 \pi n}{T} t} e^{-i \frac{2 \pi m}{T} t} dt\\ &= \begin{cases}1 & n = m \\ 0 & n \neq m \end{cases} \end{align*}\]

Once you fix a Hilbert’s space’s basis, any element in the Hilbert space can be equivalently thought of as a sequence of coefficients of the element projected onto each element of the basis:

\[f := \sum_{z=-\infty}^{\infty} \langle f, \phi_z \rangle_H \phi_z\]

with the inner product redefined as:

\[\langle f, g \rangle_H = \sum_z \langle f, \phi_z \rangle_H \langle g, \phi_z \rangle_H\]

The implication of this is that once you fix a Hilbert space’s basis, \(H\) becomes equivalent to \(\ell_2\) because all functions/vectors look the same: sequences of basis coefficients.