Abstract Linear Algebra I Unit 2 Review: Linear Independence and Bases

Key Concepts

• Linear independence a fundamental concept in linear algebra determines if vectors in a set are not linear combinations of each other
• Spanning sets generate entire vector spaces through linear combinations of their elements
• Basis a linearly independent spanning set for a vector space provides a unique representation for each vector
• Dimension the number of vectors in a basis set equals the dimension of the vector space
• Coordinate vectors represent vectors in a vector space as a linear combination of basis vectors
• Linear transformations can be represented by matrices using the basis vectors of the domain and codomain spaces
• Orthonormal bases consist of orthogonal unit vectors simplify computations and provide geometrical insights

Vector Spaces Refresher

• Vector space a set $V$ with addition and scalar multiplication operations satisfying specific axioms (closure, associativity, commutativity, identity, inverse, and distributivity)
  • Examples include $\mathbb{R}^n$, $\mathbb{C}^n$, and the set of polynomials of degree $\leq n$
• Subspace a subset of a vector space that is also a vector space under the same operations
  • Must contain the zero vector and be closed under addition and scalar multiplication
• Linear combination a sum of scalar multiples of vectors $a_1\vec{v_1} + a_2\vec{v_2} + \cdots + a_n\vec{v_n}$
• Span the set of all linear combinations of a given set of vectors
• Null space (kernel) the set of all vectors $\vec{x}$ such that $A\vec{x} = \vec{0}$ for a matrix $A$
• Column space (range) the span of the columns of a matrix $A$

Understanding Linear Independence

• Linearly independent set a set of vectors where no vector can be expressed as a linear combination of the others
  • Formally, $a_1\vec{v_1} + a_2\vec{v_2} + \cdots + a_n\vec{v_n} = \vec{0}$ implies $a_1 = a_2 = \cdots = a_n = 0$
• Linearly dependent set a set of vectors where at least one vector can be expressed as a linear combination of the others
• Trivial solution the zero vector is always a linear combination of any set of vectors with coefficients equal to zero
• Nontrivial solution a linear combination of vectors equal to the zero vector with at least one nonzero coefficient
• Unique representation property linearly independent sets allow each vector to be uniquely expressed as a linear combination of the set's vectors

Dependent vs Independent Sets

• Linearly dependent sets contain redundant information as some vectors can be expressed using others
  • Example: $\{(1, 0), (0, 1), (1, 1)\}$ is linearly dependent since $(1, 1) = 1(1, 0) + 1(0, 1)$
• Linearly independent sets contain no redundant information and provide a minimal representation of the vector space
  • Example: $\{(1, 0), (0, 1)\}$ is linearly independent in $\mathbb{R}^2$
• Testing linear independence solve the equation $a_1\vec{v_1} + a_2\vec{v_2} + \cdots + a_n\vec{v_n} = \vec{0}$ and check if the only solution is the trivial solution
  • Can be done using Gaussian elimination or by computing the determinant of the matrix formed by the vectors as columns
• Geometric interpretation in $\mathbb{R}^2$, two vectors are linearly independent if they do not lie on the same line; in $\mathbb{R}^3$, three vectors are linearly independent if they do not lie on the same plane

Spanning Sets and Their Properties

• Spanning set a set of vectors that can generate the entire vector space through linear combinations
  • Example: $\{(1, 0), (0, 1)\}$ spans $\mathbb{R}^2$ since any vector $(a, b)$ can be written as $a(1, 0) + b(0, 1)$
• Span the set of all linear combinations of a given set of vectors denoted as $\text{span}(\vec{v_1}, \vec{v_2}, \ldots, \vec{v_n})$
• Finite-dimensional vector space a vector space that can be spanned by a finite set of vectors
• Infinite-dimensional vector space a vector space that cannot be spanned by a finite set of vectors (e.g., the space of all continuous functions on $[0, 1]$)
• Redundant vectors vectors in a spanning set that can be removed without changing the span
• Minimal spanning set a spanning set with no redundant vectors

Basis: Definition and Importance

• Basis a linearly independent spanning set for a vector space
  • Provides a minimal and unique representation for each vector in the space
• Standard basis the most common basis for $\mathbb{R}^n$: $\{(1, 0, \ldots, 0), (0, 1, \ldots, 0), \ldots, (0, 0, \ldots, 1)\}$
• Coordinate vector the representation of a vector as a linear combination of basis vectors
  • Example: $(3, 2)$ in $\mathbb{R}^2$ has coordinate vector $[3, 2]^T$ with respect to the standard basis
• Change of basis converting between different bases for the same vector space
  • Allows for more convenient representations depending on the application
• Orthonormal basis a basis consisting of orthogonal unit vectors simplifies computations and provides geometrical insights

Finding and Constructing Bases

• Gaussian elimination can be used to find a basis from a spanning set by identifying and removing linearly dependent vectors
  • Nonzero rows of the reduced row echelon form correspond to basis vectors
• Gram-Schmidt process constructs an orthonormal basis from a linearly independent set of vectors
  • Iteratively orthogonalizes and normalizes vectors to create orthonormal basis vectors
• Eigenvalues and eigenvectors can be used to construct bases for certain vector spaces
  • Eigenvectors corresponding to distinct eigenvalues are linearly independent
• Polynomial bases common bases for polynomial vector spaces include the standard basis $\{1, x, x^2, \ldots, x^n\}$ and the Lagrange basis
• Fourier basis an orthonormal basis for the space of periodic functions consists of complex exponentials

Dimension of Vector Spaces

• Dimension the number of vectors in a basis set equals the dimension of the vector space
  • All bases for a vector space have the same number of vectors
• Finite-dimensional vector spaces have a finite basis and a well-defined dimension
  • Example: $\mathbb{R}^n$ has dimension $n$
• Infinite-dimensional vector spaces have no finite basis and their dimension is not well-defined
  • Example: the space of all continuous functions on $[0, 1]$
• Rank-nullity theorem for a linear transformation $T: V \to W$, $\dim(V) = \dim(\text{null}(T)) + \dim(\text{range}(T))$
• Isomorphic vector spaces vector spaces of the same dimension are isomorphic (structurally identical)

Applications and Examples

• Linear systems a basis for the solution space of a homogeneous linear system $A\vec{x} = \vec{0}$ is given by the special solutions corresponding to the free variables
• Differential equations the general solution of a homogeneous linear differential equation can be expressed as a linear combination of basis solutions
• Quantum mechanics the state of a quantum system is represented by a vector in a Hilbert space, with orthonormal bases corresponding to observable quantities
• Computer graphics points in 3D space are represented using a basis, often the standard basis or a basis determined by the camera orientation
• Cryptography certain cryptographic protocols, such as lattice-based cryptography, rely on the properties of high-dimensional vector spaces and their bases
• Data compression techniques like principal component analysis (PCA) use bases to represent high-dimensional data in a lower-dimensional space while preserving the most important information