Resume
Research
Learning
Blog
Teaching
Jokes
Kernel Papers

# Hilbert Space

A Hilbert space $$\mathcal{H}$$ is an inner product space that is separable and -complete. Separable means and complete means that the limits of all Cauchy Sequences are also in $$\mathcal{H}$$ e.g. $$\mathbb{R}$$ is ok, but $$\mathbb{Q}$$ is not.

## Example Hilbert Spaces

• The finite-dimensional reals $$\mathbb{R}^D$$ with inner product $$\langle u, v \rangle := \sum_{d=1}^D u_d v_d^*$$

• $$\ell^2 := \{ u \in \mathbb{R}^{\mathbb{N}} : \sum_n u_n^2 < \infty$$ with inner product $$\langle u, v \rangle := \sum_{n} u_n v_n^*$$

• $$L^2([0, T]) := \{ f: X \rightarrow \mathbb{C} : \int_0^T |f(x)|^2 dx < \infty$$ with inner product $$\langle u, v \rangle := \int u(x) v(x)^* dx$$

## Properties

• $$L^2$$ is the only Hilbert space of the $$L^p$$ spaces.

• Any Hilbert space possess an orthonormal basis $$\{ \phi_n \}_{n \geq 1}$$ which satisfies $$\langle \phi_n, \phi_m \rangle = \delta_{n, m}$$, where $$\delta_{n, m}$$ is the Kronecker delta.

• A consequence of Hilbert spaces possessing orthonormal bases is that any function in a space can be expressed as a linear combination of that space’s basis.

• All elements of a Hilbert space can create linear functionals in that space. Specifically, if $$f \in \mathcal{H}$$, then we can define a functional $$F: \mathcal{H} \rightarrow \mathbb{C}$$ as $$F_f(g) := \langle f, g \rangle$$

• Relatedly, for any continuous linear functional $$F: \mathcal{H} \rightarrow \mathbb{C}$$, there exists an element $$f$$ in the Hilbert space such that the functional is given by the inner product: $$F(g) = \langle f , g \rangle$$

Continuity is necessary! Otherwise, this doesn’t work. For example, suppose we want to find $$f$$ to construct the functional $$F_f(g) := g(x)$$. We know that we must be able to write this functional as an inner product $$\int g(t) f(t)^* dt$$, but the only function that can result in $$g(x) = \int g(t) f(t)^* dt$$ is the Dirac delta function $$\delta(x)$$, but it doesn’t belong to the Hilbert space!

## Bases for Hilbert Spaces

One common orthonormal basis for the Hilbert space $$L^2([0, T])$$ is the Fourier basis, defined as:

$\phi_z(t) := \frac{1}{\sqrt{T}} e^{i \frac{2 \pi z}{T} t}$

This basis is orthonormal because

\begin{align*} \langle \phi_n, \phi_m \rangle_{\mathcal{H}} &= \frac{1}{T} \int_{0}^T e^{i \frac{2 \pi n}{T} t} e^{-i \frac{2 \pi m}{T} t} dt\\ &= \begin{cases}1 & n = m \\ 0 & n \neq m \end{cases} \end{align*}

Once you fix a Hilbert’s space’s basis, any element in the Hilbert space can be equivalently thought of as a sequence of coefficients of the element projected onto each element of the basis:

$f := \sum_{z=-\infty}^{\infty} \langle f, \phi_z \rangle_H \phi_z$

with the inner product redefined as:

$\langle f, g \rangle_H = \sum_z \langle f, \phi_z \rangle_H \langle g, \phi_z \rangle_H$

The implication of this is that once you fix a Hilbert space’s basis, $$H$$ becomes equivalent to $$\ell_2$$ because all functions/vectors look the same: sequences of basis coefficients.