Bessel’s Inequality
In this tutorial, we will briefly establish the important result of Bessel’s Inequality and the consequence that the orthonormal eigenfunction expansion induced by a given Hermitian operator will converge in the so-called \( L^2 \) sense. Particularly, we consider the partial sum of the eigenfunction expansion:
\begin{align}
S_p[f] = \sum_{j=1}^{p} \langle f, \varphi_j \rangle \varphi_j \tag{1}
\end{align}
that is, the first \( p \) terms of the full series. The subsequent steps follow from this StackExchange post. Then, by orthogonality:
\begin{align}
\Vert S_p[f] \Vert^2 = \langle \sum_{i=1}^{p} \langle f, \varphi_i \rangle \varphi_i, \sum_{j=1}^{p} \langle f, \varphi_j \rangle \varphi_j \rangle = \sum_{j=1}^{p} | \langle f, \varphi_j \rangle |^2 \tag{2}
\end{align}
because \( \langle \varphi_i, \varphi_j \rangle = 0 \) when \( i \neq j \) and \( \langle \varphi_i, \varphi_j \rangle = 1 \) when \( i = j \). And,
\begin{align}
\langle f, S_p[f] \rangle &= \langle f, \sum_{j=1}^{p} \langle f, \varphi_j \rangle \varphi_j \rangle\\
&= \sum_{j=1}^{p} \overline{\langle f, \varphi_j \rangle} \langle f, \varphi_j \rangle \\
&= \sum_{j=1}^{p} | \langle f, \varphi_j \rangle |^2 = \Vert S_p[f] \Vert^2 \tag{3}
\end{align}
Therefore, by the Cauchy-Schwarz Inequality (see (5) of this tutorial), we then have Bessel’s Inequality as
\begin{align}
\Vert S_p[f] \Vert^2 = \langle f, S_p[f] \rangle &\leq \Vert f \Vert \Vert S_p[f] \Vert \\
\Vert S_p[f] \Vert &\leq \Vert f \Vert \Leftrightarrow \Vert S_p[f] \Vert^2 \leq \Vert f \Vert^2 \tag{4}
\end{align}
The implication is that the partial sum of squares of eigenfunction coefficients is bounded above by the norm of the function itself.
Parseval’s Identity
The next step is to derive Parseval’s Identity. \( L^2 \) convergence for the partial sum of the expansion in a complete orthonormal basis means that
\begin{align}
\lim_{p\to\infty} \Vert f -S_p[f] \Vert^2 = 0 \tag{5}
\end{align}
Since
\begin{align}
\langle f, S_p[f] \rangle &= \langle S_p[f], S_p[f] \rangle \\
\langle f -S_p[f], S_p[f] \rangle &= 0 \tag{6}
\end{align}
according to (3), so by Pythagoras’ Theorem:
\begin{align}
\Vert f \Vert^2 &= \Vert f -S_p[f] + S_p[f] \Vert^2 \\
\Vert f \Vert^2 &= \Vert f -S_p[f] \Vert^2 + \Vert S_p[f] \Vert^2 \\
\lim_{p\to\infty} \Vert f \Vert^2 &= \lim_{p\to\infty} (\Vert f -S_p[f] \Vert^2 + \Vert S_p[f] \Vert^2) \\
\Vert f \Vert^2 &= (0) + \lim_{p\to\infty} \sum_{j=1}^{p} | \langle f, \varphi_j \rangle |^2 && \text{(By (2) and (5))} \\
\Rightarrow \Vert f \Vert^2 &= \sum_{j=1}^{\infty} | \langle f, \varphi_j \rangle |^2 \tag{7}
\end{align}
We note that at (finitely many) jump discontinuities, the Fourier series will converge to the average of the left and right limits there.
Best Approximation by Fourier Series
The partial sum (1) adapted to the Fourier expansion ((6) of the last tutorial) will then be the best approximation of the function \( f \) using sines and cosines up to order \( p \) with respect to the \( L^2 \) distance, a.k.a. the best-fit trigonometric (Fourier) polynomial of degree \(p\). This is known as the Best Approximation Theorem: Such a Fourier partial sum \( S_p[f] \) is essentially the orthogonal projection of \( f \) onto the subspace spanned by the corresponding orthonormal Fourier basis up to order \( p \), and an orthogonal projection always achieves the minimum distance. Hence, the higher the degree \( p \) of the partial sum of the Fourier expansion, the closer the approximation to the given function. This can also be seen heuristically from (2) and Bessel’s Inequality/Parseval’s Identity: as \( p \) increases, the “energy” \( \Vert S_p[f] \Vert^2 = \sum_{j=1}^{p} | \langle f, \varphi_j \rangle |^2 \) contributed by all the coefficients increases monotonically and approaches the upper bound \( \Vert f \Vert^2 \) of the actual function.
Below shows an illustration of the linear function \( x = (2\pi/N)t -\pi \) rescaled and the partial sums of its Fourier series up to degrees 3, 10, 20 respectively. (Refer to the example in the last post) We can see that as the degree \( p \) goes up, \( S_p[f] \) becomes a better approximation to the straight line. Eventually, as \( p \to \infty \) and the expression tends to the full Fourier series, \( S_p[f] \) will converge to \( f \) in the \(L^2\) sense as suggested above.
Exercise
Show that for two \( L^2 \)-integrable function \( f \) and \( g \), i.e. \( \Vert f \Vert^2 = \int |f(x)|^2 dx < \infty, \Vert g \Vert^2 = \int |g(x)|^2 dx < \infty \) under the \( L^2 \) inner product, their product \( fg \) is \( L^1 \)-integrable, i.e. \( \langle f, g \rangle = \int |f(x)g(x)| dx < \infty \).
Answer
This follows directly from the Cauchy-Schwarz Inequality ((5) in this tutorial):
\begin{align}
\int |f(x)g(x)| dx = \int |f(x)||g(x)| dx &= \langle |f|,|g| \rangle \\
&\leq \Vert |f| \Vert \Vert |g| \Vert \\
&= (\int |f(x)|^2 dx)^{\frac{1}{2}} (\int |g(x)|^2 dx)^{\frac{1}{2}} \\
&= \Vert f \Vert \Vert g \Vert < \infty
\end{align}
Leave a Reply