Benjamin's Maths World

Spectral Theorem for Hermitian Operators

Perhaps the most significant and profound result about Hermitian operators is the Spectral Theorem, which we now state below in a suitable context.

Theorem 1. (Spectral Theorem for Hermitian Operators) For a compact, Hermitian operator \( \mathcal{L}: \mathcal{V} \to \mathcal{V} \), where \( \mathcal{L}^\dagger = \mathcal{L} \) and \( \mathcal{V} \) is a separable (and infinite-dimensional) Hilbert space, denote

1. its eigenvalues by \( \lambda_j \) and the corresponding orthonormal (see Properties 2 of the last post) eigenfunctions by \( \varphi_j \);
2. The orthogonal projection onto the eigenfunction by \( T_j(f) = \langle f, \varphi_j \rangle \)

then we have:

1. \( f = \sum_{j=1}^{\infty} \langle f, \varphi_j \rangle \varphi_j = \langle f, \varphi_1 \rangle \varphi_1 + \langle f, \varphi_2 \rangle \varphi_2 + \cdots \). This is known as the eigenfunction expansion of \(f\); or equivalently in a shorthand notation, \( 1 = \sum_{j=1}^{\infty} T_j = T_1 + T_2 + \cdots \);
2. \( \mathcal{L} = \sum_{j=1}^{\infty} \lambda_j T_j = \lambda_1T_1 + \lambda_2T_2 + \cdots \)

Item 1 implies that a Hermitian operator induces a complete orthonormal basis formed by the countably infinite eigenfunctions \( \{ \varphi_j \} \), for the separable Hilbert space it resides in. Hence any given function \( f \) can be written as an infinite sum/linear combination of these eigenfunctions. To establish this is not trivial and requires getting involved with Functional Analysis, which is out of our scope. Another caveat is the requirement of compactness, but it can be salvaged by considering the resolvent in the context of Sturm-Liouville operators. (see this StackExchange post) Additionally, given the \(L^2\)-distance, such an infinite sum also converges in an \(L^2\) sense. Item 2 can be heuristically shown in the following fashion, given a function \( f = c_1 \varphi_1 + c_2 \varphi_2 + \cdots = \sum_{j=1}^{\infty} c_j \varphi_j \) expanded according to Item 1:

\begin{align}
\mathcal{L}[f] &= \mathcal{L}[c_1 \varphi_1 + c_2 \varphi_2 + \cdots] = \mathcal{L}[\sum_{j=1}^{\infty} c_j \varphi_j] \\
&= c_1 \mathcal{L}[\varphi_1] +c_2 \mathcal{L}[\varphi_2] + \cdots = \sum_{j=1}^{\infty} c_j \mathcal{L}[\varphi_j] \\
&= \lambda_1 c_1\varphi_1 + \lambda_2 c_2\varphi_2 + \cdots = \sum_{j=1}^{\infty} \lambda_j c_j \varphi_j \\
&= \lambda_1 T_1[f] + \lambda_2 T_2[f] + \cdots \\
&= (\lambda_1 T_1 + \lambda_2 T_2 + \cdots)[f] = (\sum_{j=1}^{\infty} \lambda_j T_j) [f] \tag{1}
\end{align}

In the third line, we recall the definition of an eigenfunction, and in the fourth line, we just recognize orthogonal projections. Notice a caveat in the second line where we have passed the Hermitian operator \( \mathcal{L} \) into the infinite sum. This only works if it is compact, but again it can be fixed by the remedy mentioned above, as long as the R.H.S. actually converges.

Fourier Basis

Now let’s consider the separable Hilbert space \( L^2[-\pi, \pi] \), that is, of square-integrable functions along \( −\pi ≤ x ≤ \pi \). In (8) of this tutorial, we have shown that \( y^{\prime\prime} + \lambda y = 0 \) is a Sturm-Liouville equation with the corresponding Sturm-Liouville operator \( \mathcal{L} = -d^2/dx^2\), and the possible solutions consist of \( \sin(\sqrt{\lambda} x), \cos(\sqrt{\lambda} x)\) where \( \lambda > 0 \). As we already know, this \( \mathcal{L} \) will be Hermitian if the corresponding boundary conditions are fulfilled. These can be found by formally carrying out integration by parts:

\begin{align}
\langle \mathcal{L}[f] , g \rangle &= \int_{-\pi}^{\pi} \mathcal{L}[f(x)]\overline{g(x)} dx \\
&= \int_{-\pi}^{\pi} \frac{d^2}{dx^2}(f(x))\overline{g(x)} dx \\
&= [\frac{df(x)}{dx} \overline{g(x)}]_{-\pi}^{\pi} -\int_{-\pi}^{\pi} \frac{d}{dx}(f(x)) \frac{d}{dx}(\overline{g(x)}) dx \\
&= [\frac{df(x)}{dx} \overline{g(x)}]_{-\pi}^{\pi} -[f(x) \frac{d\overline{g(x)}}{dx}]_{-\pi}^{\pi} \\ &\quad +\int_{-\pi}^{\pi} f(x) \frac{d^2}{dx^2}(\overline{g(x)}) dx \\
&= [\frac{df(x)}{dx} \overline{g(x)}]_{-\pi}^{\pi} -[f(x) \frac{d\overline{g(x)}}{dx}]_{-\pi}^{\pi} \\ &\quad +\int_{-\pi}^{\pi} f(x) \overline{\frac{d^2}{dx^2}(g(x))} dx \\
&= [\frac{df(x)}{dx} \overline{g(x)}]_{-\pi}^{\pi} -[f(x) \frac{d\overline{g(x)}}{dx}]_{-\pi}^{\pi} \\ &\quad +\int_{-\pi}^{\pi} f(x) \overline{\mathcal{L}[g(x)]} dx \tag{2}
\end{align}

This shows that \(\mathcal{L}^{\dagger} = \mathcal{L} = d^2/dx^2\) is self-adjoint, and the boundary conditions for Hermicity are

\begin{align}
[\frac{df(x)}{dx} \overline{g(x)}]_{-\pi}^{\pi} -[f(x) \frac{d\overline{g(x)}}{dx}]_{-\pi}^{\pi} = 0 \tag{3}
\end{align}

For these two boundary terms to vanish, the eigenfunctions \( \sin(\sqrt{\lambda} x), \cos(\sqrt{\lambda} x)\) should have \( m = \sqrt{\lambda} = 0, 1, 2, \ldots \) as a non-negative integer. Hence a countably infinite, orthogonal basis of eigenfunctions induced by \(\mathcal{L}\) for the \( L^2[-\pi, \pi] \) space are \(\{\sin(x), \sin(2x), \sin(3x), \ldots, 1, \cos(x),\cos(2x), \cos(3x), \ldots\}\). This basis, consisting of sines and cosines with discrete, equally spaced frequencies, is then famously known as the Fourier basis.

Conventionally, we will express the Fourier basis as an orthonormal one through dividing each eigenfunction by its norm, \( 1/\sqrt{\pi} \) for each of them (\( 1/ \sqrt{2\pi}\) for \(1\)): Below we will show that for the cosines, but the norm calculation is the same for the sines. For any integer \( m \),

\begin{align}
\Vert \cos(mx) \Vert^2 &= \int_{-\pi}^{\pi} \cos(mx) \overline{\cos(mx)} dx \\
&= \int_{-\pi}^{\pi} \cos^2(mx) dx \\
&= \int_{-\pi}^{\pi} \frac{1}{2}(1 + \cos(2mx)) dx \\
&= [\frac{1}{2}x + \frac{1}{4m} \sin(2mx)]_{-\pi}^{\pi} = \frac{1}{2}(2\pi) + (0) = \frac{1}{\pi} \tag{4}
\end{align}

and hence the new orthonormal Fourier basis is

\begin{align}
\{\frac{1}{\sqrt{\pi}}\sin(x), \frac{1}{\sqrt{\pi}}\sin(2x), \frac{1}{\sqrt{\pi}}\sin(3x), \ldots, \\
\frac{1}{\sqrt{2\pi}}, \frac{1}{\sqrt{\pi}}\cos(x), \frac{1}{\sqrt{\pi}}\cos(2x), \frac{1}{\sqrt{\pi}}\cos(3x), \ldots\} \tag{5}
\end{align}

Fourier Series

After obtaining the Fourier basis, by Item 1 of the Spectral Theorem above, we can proceed to compute the so-called Fourier Series of any \( L^2 \)-integrable function \( f(x) \) defined over \( [-\pi, \pi] \). It is often expressed in the form of

\begin{align}
f(x) &= \frac{a_0}{2} + a_1 \cos(x) + a_2 \cos(2x) + a_3 \cos(3x) + \cdots \\
&\quad + b_1 \sin(x) + b_2 \sin(2x) + b_3 \sin(3x) + \cdots \\
&= \frac{a_0}{2} + \sum_{m=1}^{\infty} a_m\cos(mx) + \sum_{m=1}^{\infty} b_m\sin(mx) \tag{6}
\end{align}

The Fourier coefficients \( a_m, b_m \) are then found by comparing (6) with that indicated in the Spectral Theorem applying over (5):

\begin{align}
f &= \sum_{j=1}^{\infty} \langle f, \varphi_j \rangle \varphi_j \\
&= \langle f, \frac{1}{\sqrt{2\pi}} \rangle \frac{1}{\sqrt{2\pi}} \\
&\quad +\langle f, \frac{1}{\sqrt{\pi}}\cos(x) \rangle (\frac{1}{\sqrt{\pi}}\cos(x)) +\langle f, \frac{1}{\sqrt{\pi}} \cos(2x) \rangle( \frac{1}{\sqrt{\pi}}\cos(2x)) + \cdots \\
&\quad +\langle f, \frac{1}{\sqrt{\pi}} \sin(x) \rangle (\frac{1}{\sqrt{\pi}}\sin(x)) +\langle f, \frac{1}{\sqrt{\pi}}\sin(2x) \rangle (\frac{1}{\sqrt{\pi}}\sin(2x)) + \cdots \\
&= \frac{1}{2\pi}\langle f, 1 \rangle + \frac{1}{\pi} \langle f, \cos(x) \rangle \cos(x) + \frac{1}{\pi} \langle f, \cos(2x) \rangle \cos(2x) + \cdots \\
&\quad + \frac{1}{\pi} \langle f, \sin(x) \rangle \sin(x) + \frac{1}{\pi} \langle f, \sin(2x) \rangle \sin(2x) + \cdots \tag{7}
\end{align}

Therefore, we have the following formulae:

\begin{align}
a_m &= \frac{1}{\pi} \langle f, \cos(mx) \rangle = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x) \cos(mx) dx \tag{8}\\
b_m &= \frac{1}{\pi} \langle f, \sin(mx) \rangle = \frac{1}{\pi} \int_{-\pi}^{\pi} f(x) \sin(mx) dx \tag{9}
\end{align}

For other intervals that are not \( [-\pi, \pi] \), let’s say \( [a,b] \), then we just need to make a linear change of variable \( t = 2\pi(t-a)/(b-a) -\pi \) and proceed as usual.

Example

Now we will demonstrate this by expanding the function \( x \) into a Fourier series, over the standard interval \( [-\pi, \pi] \). It is simply to compute \( a_m \) and \( b_m\) mechanically according to (8) and (9). However, note that \(x\) is an odd function while cosines are even, the integrals for \( a_m \) where the resulting integrands are then odd, within the symmetric interval over the origin, are thus all zero. Subsequently, we just need to calculate the general form of \( b_m\) as below:

\begin{align}
b_m &= \frac{1}{\pi}\int_{-\pi}^{\pi} x \sin(mx) dx \\
&= \frac{1}{\pi}[-\frac{x}{m} \cos(mx)]_{-\pi}^{\pi} -\frac{1}{\pi} \int_{-\pi}^{\pi} (-\frac{1}{m}\cos(mx)) dx \\
&= \frac{1}{\pi}[-\frac{\pi}{m} \cos(m\pi) + \frac{(-\pi)}{m}\cos(m(-\pi))] \\
&\quad +\frac{1}{m\pi} \int_{-\pi}^{\pi} \cos(mx) dx \\
&= -\frac{1}{m}\cos(m\pi) -\frac{1}{m} \cos(-m\pi) + \frac{1}{m^2\pi}[\sin(mx)]_{-\pi}^{\pi} \\
&= -\frac{2(-1)^m}{m} + (0) = -\frac{2(-1)^m}{m} \tag{10}
\end{align}

as \( \cos(m\pi) = \cos(-m\pi) = (-1)^m \). Therefore, the Fourier series of \(x\) over \( [-\pi, \pi] \) is

\begin{align}
x = -2\sum_{m=1}^{\infty} \frac{(-1)^m}{m} \sin(mx) \tag{11}
\end{align}

Fourier Sine/Cosine Series

For an odd/even function like \( x \) in the above example, due to the symmetry, we may consider only either the set of sines or cosines over the half-interval \( [0, \pi] \) and the mirrored results will be the same. In a reverse direction, we may define a Fourier sine/cosine series for a function over some fixed interval, let’s say also \( [0, \pi] \), by using just sines or cosines in the original Fourier basis for the whole range \( [-\pi, \pi] \). In the above example, the Fourier sine series of \( x \) over the half-interval \( [0, \pi] \) is exactly (11).

Exercise

Find the Fourier series of \( x^2 \) over the same interval \( [-\pi, \pi] \).

Answer

This time \( x^2 \) is even, so by a similar logic as above, only the cosine components remain. We then compute \( a_m \) as suggested in (8), integrating by parts:

\begin{align}
a_m &= \frac{1}{\pi} \int_{-\pi}^{\pi} x^2 \cos(mx) dx \\
&= [\frac{1}{m\pi} x^2 \sin(mx)]_{-\pi}^{\pi} -\frac{2}{m\pi} \int_{-\pi}^{\pi} x \sin(mx) dx \\
&= (0) -\frac{2}{m\pi} [-\frac{x}{m} \cos(mx)]_{-\pi}^{\pi} + \frac{2}{m^2\pi} \int_{-\pi}^{\pi} (-\cos(mx)) dx \\
&= \frac{2}{m^2\pi}[\pi(-1)^m – (-\pi)(-1)^m] -(0) = \frac{4(-1)^m}{m^2}
\end{align}

Meanwhile, the constant term is just

\begin{align}
\frac{a_0}{2} &= \frac{1}{2\pi} \int_{-\pi}^{\pi} x^2 dx \\
&= \frac{1}{2\pi} [\frac{x^3}{3}]_{-\pi}^{\pi} \\
&= \frac{1}{2\pi} [\frac{\pi^3}{3} -\frac{(-\pi)^3}{3}] = \frac{\pi^2}{3}
\end{align}

Therefore, the desired Fourier series is

\begin{align}
x^2 = \frac{\pi^2}{3} + \sum_{m=1}^{\infty} \frac{4(-1)^m}{m^2} \cos(mx)
\end{align}