6.3 Projection operators and their properties

Projection operators in Hilbert spaces are key tools for analyzing and manipulating vectors. They allow us to map elements onto specific subspaces, preserving important properties like orthogonality and minimizing distances.

These operators have unique characteristics like idempotence and self-adjointness. They're crucial for tasks such as orthogonal decomposition and least squares approximation, making them indispensable in many areas of mathematics and physics.

Projection Operators in Hilbert Spaces

Definition of projection operators

• Linear operator $P$ on a Hilbert space $H$ satisfies idempotence property $P^2 = P$
• Satisfies self-adjointness property $P^* = P$, where $P^*$ denotes the adjoint operator of $P$
• Eigenvalues of projection operators are restricted to either 0 or 1 (no other values possible)
• Orthogonal projection operators have range and null space that are orthogonal to each other
• Identity operator can be expressed as the sum of orthogonal projection operators onto orthogonal subspaces

Construction of projection operators

• Projection operator $P_M$ onto a closed subspace $M$ of Hilbert space $H$ maps any element $x \in H$ to the unique element $P_M(x)$ in $M$ that minimizes the distance $\|x - P_M(x)\|$
  • $P_M(x)$ is the closest point in $M$ to $x$ (minimizes the norm of the difference)
• Constructing $P_M$ using an orthonormal basis $\{e_i\}_{i=1}^n$ for $M$: $P_M(x) = \sum_{i=1}^n \langle x, e_i \rangle e_i$
  • Inner products $\langle x, e_i \rangle$ give the coordinates of the projection in the basis
  • Summing the basis vectors $e_i$ scaled by these coordinates yields the projection $P_M(x)$
• Constructing $P_M$ using the orthogonal complement $M^\perp$: $P_M = I - P_{M^\perp}$, where $I$ is the identity operator
  • Subtracting the projection onto the orthogonal complement from the identity gives the projection onto the original subspace

Properties of projection operators

• Orthogonality: For any $x \in H$, the difference $x - P_M(x)$ lies in the orthogonal complement $M^\perp$
  • Proof: $\langle x - P_M(x), y \rangle = \langle x, y \rangle - \langle P_M(x), y \rangle = 0$ for any $y \in M$, since $P_M(x) \in M$
• Idempotence: For any projection operator $P$ on $H$, applying it twice yields the same result as applying it once, i.e., $P^2 = P$
  • Proof: $P^2(x) = P(P(x)) = P(x)$ for any $x \in H$, since $P(x)$ lies in the range of $P$ where $P$ acts as the identity

Applications of projection operators

• Orthogonal decomposition: Any $x \in H$ can be uniquely decomposed as $x = P_M(x) + P_{M^\perp}(x)$, where $P_M(x) \in M$ and $P_{M^\perp}(x) \in M^\perp$
  • Components $P_M(x)$ and $P_{M^\perp}(x)$ are orthogonal, i.e., $\langle P_M(x), P_{M^\perp}(x) \rangle = 0$
  • Useful for separating a vector into its components in a subspace and its orthogonal complement
• Least squares approximation: For a given $x \in H$ and a closed subspace $M$, the projection $P_M(x)$ is the best approximation of $x$ in $M$ in the least squares sense
  • Minimizes the distance $\|x - P_M(x)\|$ over all elements in $M$
  • Finds the closest point in $M$ to the given vector $x$ (useful for data fitting and regression problems)