Linear Algebra 101 Unit 4 ReviewEigenvalues and Eigenvectors

Key Concepts

• Eigenvalues represent the scaling factors of a linear transformation in the direction of the corresponding eigenvectors
• Eigenvectors are non-zero vectors that, when a linear transformation is applied, remain in the same direction or are scaled by a factor
• The eigenspace of an eigenvalue is the set of all eigenvectors associated with that eigenvalue, including the zero vector
• A matrix is diagonalizable if it has a full set of linearly independent eigenvectors that form a basis for the vector space
• The eigendecomposition of a matrix expresses it as a product of its eigenvectors and eigenvalues, providing insights into its structure and behavior
• The characteristic equation of a matrix, $det(A - \lambda I) = 0$, is used to find its eigenvalues
• The algebraic multiplicity of an eigenvalue is the number of times it appears as a root of the characteristic equation
• The geometric multiplicity of an eigenvalue is the dimension of its corresponding eigenspace

Mathematical Definition

• For a square matrix $A$, a non-zero vector $v$ is an eigenvector of $A$ if there exists a scalar $\lambda$ such that $Av = \lambda v$
  • $\lambda$ is the eigenvalue corresponding to the eigenvector $v$
• The eigenvalue equation can be rewritten as $(A - \lambda I)v = 0$, where $I$ is the identity matrix
• To find the eigenvalues of a matrix $A$, solve the characteristic equation: $det(A - \lambda I) = 0$
  • The roots of this polynomial equation are the eigenvalues of $A$
• For each eigenvalue $\lambda_i$, the corresponding eigenvectors $v_i$ are found by solving $(A - \lambda_i I)v_i = 0$
• The set of all eigenvectors corresponding to an eigenvalue $\lambda$, along with the zero vector, forms the eigenspace of $\lambda$
• A matrix $A$ is diagonalizable if it can be expressed as $A = PDP^{-1}$, where $D$ is a diagonal matrix of eigenvalues and $P$ is a matrix whose columns are the corresponding eigenvectors

Geometric Interpretation

• Eigenvectors represent the directions in which a linear transformation acts as a scaling operation
  • The corresponding eigenvalues determine the scale factor in each direction
• In a 2D space, the eigenvectors of a matrix are the axes along which the transformation stretches or compresses the space
• Eigenvectors with positive eigenvalues point in the direction of stretching, while those with negative eigenvalues point in the direction of compression
• Eigenvectors with eigenvalues greater than 1 indicate stretching, while those with eigenvalues between 0 and 1 indicate compression
• Eigenvectors with an eigenvalue of 1 remain unchanged under the transformation
• In higher dimensions, eigenvectors represent the principal axes of the transformation, along which the space is scaled according to the corresponding eigenvalues
• The eigenspace of an eigenvalue is a subspace of the vector space that is invariant under the linear transformation

Calculating Eigenvalues and Eigenvectors

• To find the eigenvalues of a matrix $A$, solve the characteristic equation: $det(A - \lambda I) = 0$
  • Expand the determinant and solve the resulting polynomial equation for $\lambda$
• For each eigenvalue $\lambda_i$, find the corresponding eigenvectors by solving $(A - \lambda_i I)v_i = 0$
  • This results in a system of linear equations that can be solved using Gaussian elimination or other methods
• The solutions to the system of equations form the eigenspace of $\lambda_i$
  • The eigenvectors are the non-zero vectors in this space
• To find a basis for the eigenspace, identify the free variables in the solution and assign them arbitrary values to generate linearly independent eigenvectors
• Normalize the eigenvectors by dividing them by their magnitudes to obtain unit eigenvectors
• Check the algebraic and geometric multiplicities of each eigenvalue to determine if the matrix is diagonalizable

Properties and Theorems

• The sum of the eigenvalues of a matrix is equal to its trace (the sum of the diagonal elements)
• The product of the eigenvalues of a matrix is equal to its determinant
• If a matrix $A$ has distinct eigenvalues, then its eigenvectors are linearly independent
• A matrix is diagonalizable if and only if the sum of the geometric multiplicities of its eigenvalues equals the dimension of the matrix
• The eigenvalues of a triangular matrix (upper or lower) are the elements on its main diagonal
• If a matrix $A$ is symmetric, then its eigenvalues are real, and its eigenvectors are orthogonal (perpendicular to each other)
• The eigenvalues of a Hermitian matrix (conjugate transpose equals itself) are real
• The eigenvalues of a unitary matrix (conjugate transpose equals its inverse) have absolute value 1
• The Spectral Theorem states that a symmetric matrix is always diagonalizable by an orthogonal matrix

Applications in Linear Transformations

• Eigenvalues and eigenvectors are used to analyze and understand the behavior of linear transformations
• The eigenvalues of a transformation matrix determine how the transformation scales vectors in different directions
  • Positive eigenvalues indicate stretching, while negative eigenvalues indicate reflection and compression
• The eigenvectors of a transformation matrix represent the principal axes along which the transformation acts as a scaling operation
• Diagonalization of a matrix using its eigenvectors and eigenvalues simplifies the computation of matrix powers and exponentials
  • $A^n = PD^nP^{-1}$ and $e^A = Pe^DP^{-1}$, where $D$ is the diagonal matrix of eigenvalues
• Eigenvalues and eigenvectors help identify invariant subspaces and fixed points of a linear transformation
• In dynamical systems, the stability of equilibrium points can be determined by analyzing the eigenvalues of the Jacobian matrix
• Principal Component Analysis (PCA) uses eigenvectors and eigenvalues to identify the directions of maximum variance in a dataset and reduce its dimensionality

Real-World Examples

• In physics, the eigenvectors of the moment of inertia tensor represent the principal axes of rotation of a rigid body
  • The corresponding eigenvalues determine the object's resistance to rotational acceleration around each axis
• In structural engineering, the eigenvectors of the stiffness matrix represent the vibration modes of a structure
  • The corresponding eigenvalues determine the natural frequencies of vibration
• In quantum mechanics, the eigenvectors of the Hamiltonian operator represent the stationary states of a quantum system
  • The corresponding eigenvalues represent the energy levels of the system
• In computer graphics, eigenvalues and eigenvectors are used for mesh smoothing and deformation
  • The eigenvectors of the Laplacian matrix of a mesh represent its principal curvature directions
• In machine learning, eigenvalues and eigenvectors are used in dimensionality reduction techniques like PCA
  • The eigenvectors of the covariance matrix of a dataset represent the principal components that capture the most variance
• In population dynamics, the eigenvalues of the Leslie matrix determine the long-term growth rate and age structure of a population
• In network analysis, the eigenvectors of the adjacency matrix or Laplacian matrix can be used for node centrality measures and community detection

Practice Problems and Solutions

1. Find the eigenvalues and eigenvectors of the matrix $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$.

   • Characteristic equation: $det(A - \lambda I) = (2 - \lambda)^2 - 1 = \lambda^2 - 4\lambda + 3 = 0$
   • Eigenvalues: $\lambda_1 = 3$, $\lambda_2 = 1$
   • Eigenvectors:
     • For $\lambda_1 = 3$: $\begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$, eigenvector: $v_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$
     • For $\lambda_2 = 1$: $\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}$, eigenvector: $v_2 = \begin{pmatrix} -1 \\ 1 \end{pmatrix}$

2. Determine if the matrix $B = \begin{pmatrix} 1 & 2 \\ 0 & 3 \end{pmatrix}$ is diagonalizable.

   • Eigenvalues: $\lambda_1 = 1$, $\lambda_2 = 3$
   • Eigenvectors:
     • For $\lambda_1 = 1$: $v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$
     • For $\lambda_2 = 3$: $v_2 = \begin{pmatrix} 2 \\ 1 \end{pmatrix}$
   • The eigenvectors are linearly independent, so $B$ is diagonalizable

3. Find the eigenvalues and eigenvectors of the matrix $C = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$.

   • Characteristic equation: $\lambda^2 + 1 = 0$
   • Eigenvalues: $\lambda_1 = i$, $\lambda_2 = -i$
   • Eigenvectors:
     • For $\lambda_1 = i$: $v_1 = \begin{pmatrix} 1 \\ -i \end{pmatrix}$
     • For $\lambda_2 = -i$: $v_2 = \begin{pmatrix} 1 \\ i \end{pmatrix}$