9.3 Orthonormal Bases in Inner Product Spaces

Orthonormal bases are the building blocks of inner product spaces. They're special sets of vectors that are both perpendicular to each other and have a length of 1, making them super useful for describing any vector in the space.

The Gram-Schmidt process is a key tool for creating these orthonormal bases. It takes any set of vectors and transforms them into an orthonormal basis, helping us understand and work with inner product spaces more easily.

Orthonormal Bases in Inner Product Spaces

Definition and Properties

• An orthonormal basis for an inner product space $V$ is a basis $\{v_1, v_2, \ldots, v_n\}$ such that each vector $v_i$ is unit length ($\|v_i\| = 1$) and the vectors are mutually orthogonal ($\langle v_i, v_j \rangle = 0$ for $i \neq j$)
  • Example: In $\mathbb{R}^2$, the standard basis vectors $\{(1, 0), (0, 1)\}$ form an orthonormal basis
• The orthonormal basis vectors span the entire inner product space $V$, meaning any vector in $V$ can be expressed as a linear combination of the orthonormal basis vectors
  • Example: In $\mathbb{R}^3$, any vector $(a, b, c)$ can be written as $a(1, 0, 0) + b(0, 1, 0) + c(0, 0, 1)$ using the standard orthonormal basis

Construction and Non-uniqueness

• The Gram-Schmidt process is an algorithm for constructing an orthonormal basis from an arbitrary basis of a finite-dimensional inner product space
  • It involves iteratively orthogonalizing and normalizing the basis vectors
• The coefficients in the linear combination of orthonormal basis vectors are unique, given by the inner products of the vector with the basis vectors
  • These coefficients are called Fourier coefficients
• The orthonormal basis is not unique, as there can be multiple orthonormal bases for the same inner product space
  • Example: In $\mathbb{R}^2$, both $\{(1, 0), (0, 1)\}$ and $\{(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}}), (\frac{-1}{\sqrt{2}}, \frac{1}{\sqrt{2}})\}$ are orthonormal bases

Existence and Uniqueness of Orthonormal Bases

Existence Proof

• Given a finite-dimensional inner product space $V$, the Gram-Schmidt process can be applied to any basis of $V$ to construct an orthonormal basis, proving the existence of an orthonormal basis for $V$
  • The Gram-Schmidt process starts with an arbitrary basis and iteratively orthogonalizes and normalizes the vectors
  • The resulting set of vectors forms an orthonormal basis for the inner product space

Uniqueness up to Isometry

• If $\{v_1, v_2, \ldots, v_n\}$ and $\{w_1, w_2, \ldots, w_n\}$ are two orthonormal bases for a finite-dimensional inner product space $V$, then there exists an isometry (a linear map that preserves inner products) $T : V \to V$ such that $T(v_i) = w_i$ for all $i$
  • This isometry $T$ is unique and can be represented by an orthogonal matrix (a square matrix $Q$ such that $QQ^T = Q^TQ = I$)
  • The uniqueness of the isometry $T$ implies that the orthonormal basis is unique up to isometry, meaning any two orthonormal bases are related by an isometry
• Example: In $\mathbb{R}^2$, the isometry that maps the standard basis $\{(1, 0), (0, 1)\}$ to the rotated basis $\{(\cos \theta, \sin \theta), (-\sin \theta, \cos \theta)\}$ is a rotation matrix

Vector Representation with Orthonormal Bases

Fourier Coefficients

• Given an orthonormal basis $\{v_1, v_2, \ldots, v_n\}$ for an inner product space $V$ and a vector $x$ in $V$, $x$ can be expressed as a linear combination of the orthonormal basis vectors: $x = \langle x, v_1 \rangle v_1 + \langle x, v_2 \rangle v_2 + \ldots + \langle x, v_n \rangle v_n$
  • The coefficients $\langle x, v_i \rangle$ in the linear combination are the inner products of the vector $x$ with the orthonormal basis vectors $v_i$, also known as the Fourier coefficients
  • Example: In $\mathbb{R}^3$, if $x = (1, 2, 3)$ and the orthonormal basis is the standard basis, then the Fourier coefficients are $\langle x, (1, 0, 0) \rangle = 1$, $\langle x, (0, 1, 0) \rangle = 2$, and $\langle x, (0, 0, 1) \rangle = 3$

Parseval's Identity

• Parseval's identity states that for any vector $x$ in $V$, $\|x\|^2 = |\langle x, v_1 \rangle|^2 + |\langle x, v_2 \rangle|^2 + \ldots + |\langle x, v_n \rangle|^2$, relating the norm of the vector to the sum of the squares of its Fourier coefficients
  • This identity shows that the energy of a vector (square of its norm) is equal to the sum of the energies of its projections onto the orthonormal basis vectors
• The linear combination of orthonormal basis vectors provides a unique representation of any vector in the inner product space
  • This representation is useful in various applications, such as signal processing and quantum mechanics

Orthonormal Bases vs Isometries

Isometries and Orthonormal Bases

• An isometry is a linear map $T : V \to W$ between inner product spaces $V$ and $W$ that preserves inner products, i.e., $\langle T(x), T(y) \rangle = \langle x, y \rangle$ for all $x, y$ in $V$
  • Example: A rotation in $\mathbb{R}^2$ is an isometry, as it preserves the dot product between vectors
• If $\{v_1, v_2, \ldots, v_n\}$ is an orthonormal basis for $V$ and $\{w_1, w_2, \ldots, w_n\}$ is an orthonormal basis for $W$, then a linear map $T : V \to W$ defined by $T(v_i) = w_i$ for all $i$ is an isometry
  • The matrix representation of this isometry $T$ with respect to the orthonormal bases is an orthogonal matrix

Orthonormal Bases and Isometries

• Conversely, if $T : V \to W$ is an isometry and $\{v_1, v_2, \ldots, v_n\}$ is an orthonormal basis for $V$, then $\{T(v_1), T(v_2), \ldots, T(v_n)\}$ is an orthonormal basis for $W$
  • This property shows that isometries map orthonormal bases to orthonormal bases
• The relationship between orthonormal bases and isometries allows for the study of geometric properties of inner product spaces using linear algebra techniques
  • For example, the classification of isometries in $\mathbb{R}^2$ and $\mathbb{R}^3$ can be done using orthonormal bases and orthogonal matrices