🧐Functional Analysis Unit 5 – Hilbert Spaces and Orthonormal Bases

Key Concepts and Definitions

• Hilbert spaces generalize the notion of Euclidean space to infinite-dimensional vector spaces while preserving the structure of an inner product
• Complete inner product spaces where every Cauchy sequence converges to a point within the space
• Separable Hilbert spaces contain a countable dense subset (rational numbers in $\mathbb{R}$)
  • Non-separable Hilbert spaces do not have a countable dense subset ($L^2$ space of square-integrable functions)
• Bounded linear operators map elements of a Hilbert space to another Hilbert space while preserving linearity and boundedness
• Adjoint operators $A^*$ satisfy $\langle Ax, y \rangle = \langle x, A^*y \rangle$ for all $x, y$ in the Hilbert space
• Compact operators map bounded sets to relatively compact sets (finite-dimensional subspaces)
• Self-adjoint operators satisfy $A = A^*$ and have real eigenvalues

Properties of Hilbert Spaces

• Completeness ensures that limits of Cauchy sequences exist within the space
• The parallelogram law holds: $\|x+y\|^2 + \|x-y\|^2 = 2(\|x\|^2 + \|y\|^2)$ for all $x, y$ in the Hilbert space
• The polarization identity expresses the inner product in terms of norms: $\langle x, y \rangle = \frac{1}{4}(\|x+y\|^2 - \|x-y\|^2)$
• Riesz representation theorem states that every bounded linear functional on a Hilbert space can be represented as an inner product with a unique vector
• Orthogonal projection theorem guarantees the existence and uniqueness of the best approximation of a vector by an element of a closed subspace
  • The best approximation is the orthogonal projection onto the subspace
• Parseval's identity relates the norm of a vector to the sum of the squares of its Fourier coefficients: $\|x\|^2 = \sum_{n=1}^{\infty} |\langle x, e_n \rangle|^2$
• Every Hilbert space has an orthonormal basis (Gram-Schmidt process)

Inner Products and Norms

• Inner products are sesquilinear, conjugate symmetric, and positive definite
  • Sesquilinear: linear in the first argument and conjugate linear in the second
  • Conjugate symmetric: $\langle x, y \rangle = \overline{\langle y, x \rangle}$
  • Positive definite: $\langle x, x \rangle \geq 0$ with equality if and only if $x = 0$
• Norms measure the length or size of vectors and satisfy positivity, homogeneity, and the triangle inequality
• The Cauchy-Schwarz inequality bounds the inner product: $|\langle x, y \rangle| \leq \|x\| \|y\|$
  • Equality holds if and only if $x$ and $y$ are linearly dependent
• The norm induced by the inner product is defined as $\|x\| = \sqrt{\langle x, x \rangle}$
• Parallelogram law relates the norms of the sum and difference of two vectors to their individual norms

Orthogonality and Orthonormal Sets

• Orthogonality means two vectors have a zero inner product: $\langle x, y \rangle = 0$
• Orthogonal vectors are perpendicular and do not share any common components
• Orthogonal sets consist of pairwise orthogonal vectors: $\langle x_i, x_j \rangle = 0$ for $i \neq j$
• Orthonormal sets are orthogonal and have unit norm: $\|x_i\| = 1$ for all $i$
  • Obtained by normalizing orthogonal vectors: $e_i = \frac{x_i}{\|x_i\|}$
• Orthonormal sets are linearly independent and can serve as bases for subspaces or the entire Hilbert space
• Gram-Schmidt process constructs an orthonormal set from a linearly independent set by iteratively subtracting projections and normalizing

Orthonormal Bases and Their Significance

• Orthonormal bases provide a convenient way to represent vectors as linear combinations of basis elements
• Every vector can be uniquely expressed as $x = \sum_{n=1}^{\infty} \langle x, e_n \rangle e_n$, where $\{e_n\}$ is an orthonormal basis
• Fourier coefficients $\langle x, e_n \rangle$ measure the contribution of each basis element to the vector
• Parseval's identity relates the norm of a vector to its Fourier coefficients: $\|x\|^2 = \sum_{n=1}^{\infty} |\langle x, e_n \rangle|^2$
  • Generalizes Pythagoras' theorem to infinite-dimensional spaces
• Orthonormal bases simplify computations and provide a natural way to decompose functions and signals
• Examples of orthonormal bases include trigonometric functions (Fourier basis) and wavelets
• Orthonormal bases are not unique, but they all yield the same representation of vectors

Fourier Series and Expansions

• Fourier series represent periodic functions as infinite sums of trigonometric functions (sines and cosines)
• Fourier coefficients $a_n$ and $b_n$ measure the contribution of each frequency component to the function
  • $a_n = \frac{2}{T} \int_{-T/2}^{T/2} f(t) \cos(\frac{2\pi nt}{T}) dt$
  • $b_n = \frac{2}{T} \int_{-T/2}^{T/2} f(t) \sin(\frac{2\pi nt}{T}) dt$
• Fourier series converge to the function in the $L^2$ sense (mean square convergence)
• Fourier transforms extend the concept to non-periodic functions by using a continuous spectrum of frequencies
• Fourier analysis decomposes functions into their frequency components, enabling filtering and signal processing
• Parseval's theorem for Fourier series relates the $L^2$ norm of a function to its Fourier coefficients: $\int_{-T/2}^{T/2} |f(t)|^2 dt = \frac{a_0^2}{2} + \sum_{n=1}^{\infty} (a_n^2 + b_n^2)$

Applications in Functional Analysis

• Quantum mechanics uses Hilbert spaces to model the state space of quantum systems
  • Wave functions are elements of a Hilbert space, and observables are self-adjoint operators
• Signal processing relies on Fourier analysis to filter and transform signals
  • Removing noise, compressing data, and extracting features
• Partial differential equations can be solved using Hilbert space methods
  • Weak solutions are obtained by projecting onto appropriate function spaces
• Machine learning and data analysis use Hilbert space techniques for dimensionality reduction and feature extraction
  • Principal component analysis (PCA) and kernel methods
• Operator theory studies the properties and classification of linear operators on Hilbert spaces
  • Spectral theory, functional calculus, and operator algebras
• Wavelets provide localized orthonormal bases for efficient representation and analysis of signals and images

Common Pitfalls and Tips

• Ensure that the inner product is well-defined and satisfies the required properties
• Be cautious when dealing with unbounded operators, as they may not be defined on the entire Hilbert space
• Verify that a set of vectors is complete before claiming it is an orthonormal basis
• Remember that convergence in Hilbert spaces is typically in the norm topology, not pointwise convergence
• Use the appropriate form of Parseval's identity or theorem depending on the context (discrete or continuous)
• Be aware of the differences between separable and non-separable Hilbert spaces, as some results may not hold in the non-separable case
• Exploit the properties of orthonormal bases to simplify computations and proofs whenever possible
• Understand the relationship between Hilbert spaces and other function spaces, such as $L^p$ spaces and Sobolev spaces