Linear Algebra 101 Unit 3 ReviewVector Spaces

Key Concepts and Definitions

• Vector spaces consist of a set of vectors and two operations (addition and scalar multiplication) that satisfy certain axioms
• Vectors are elements of a vector space represented as ordered tuples or arrays of numbers (components)
• Scalars are real or complex numbers used to multiply vectors
• Zero vector has all components equal to zero and serves as the identity element for vector addition
• Linear combinations involve multiplying vectors by scalars and adding the results to create new vectors within the vector space
• Span of a set of vectors includes all possible linear combinations of those vectors forming a subspace
• Linear independence means a set of vectors cannot be expressed as linear combinations of each other (unique representation)
• Basis is a linearly independent set of vectors that spans the entire vector space providing a coordinate system

Vector Space Axioms

• Closure under addition states that the sum of any two vectors in the space results in another vector within the same space
• Associativity of addition allows regrouping of vector additions without changing the result: $(u + v) + w = u + (v + w)$
• Commutativity of addition means the order of vector addition does not affect the result: $u + v = v + u$
• Additive identity is the zero vector which when added to any vector results in the original vector: $v + 0 = v$
• Additive inverses exist for every vector $v$ such that $v + (-v) = 0$
• Closure under scalar multiplication guarantees that multiplying a vector by a scalar yields a vector within the same space
• Distributivity of scalar multiplication over vector addition: $a(u + v) = au + av$
• Distributivity of scalar multiplication over scalar addition: $(a + b)v = av + bv$
• Compatibility of scalar multiplication with field multiplication: $a(bv) = (ab)v$
• Scalar multiplicative identity states multiplying a vector by 1 does not change the vector: $1v = v$

Subspaces and Span

• Subspaces are smaller vector spaces contained within a larger vector space inheriting the same axioms and operations
• Subspaces must include the zero vector and be closed under vector addition and scalar multiplication
• The span of a set of vectors $\{v_1, v_2, ..., v_n\}$ is the set of all possible linear combinations of those vectors
  • Span forms a subspace of the original vector space
  • Vectors in the spanning set are called generators of the subspace
• Row space of a matrix is the span of its row vectors forming a subspace of $\mathbb{R}^n$
• Column space of a matrix is the span of its column vectors forming a subspace of $\mathbb{R}^m$
• Null space (kernel) of a matrix is the set of all vectors $x$ such that $Ax = 0$ forming a subspace of $\mathbb{R}^n$

Linear Combinations and Linear Independence

• Linear combination of vectors $\{v_1, v_2, ..., v_n\}$ is a sum of the form $a_1v_1 + a_2v_2 + ... + a_nv_n$ where $a_i$ are scalars
• Vectors are linearly dependent if one can be expressed as a linear combination of the others
  • Linearly dependent set contains redundant information
• Vectors are linearly independent if no vector can be written as a linear combination of the others
  • Unique representation for each vector in the set
• A set of vectors is linearly independent if and only if the zero vector can be expressed uniquely as the trivial linear combination (all scalars equal to zero)
• Linearly independent sets are crucial for constructing basis and understanding the dimension of vector spaces

Basis and Dimension

• A basis is a linearly independent set of vectors that spans the entire vector space
  • Minimal spanning set with no redundancy
• Dimension of a vector space is the number of vectors in its basis
  • All bases of a vector space have the same number of vectors
• Standard basis for $\mathbb{R}^n$ consists of unit vectors $\{e_1, e_2, ..., e_n\}$ where $e_i$ has a 1 in the $i$-th position and zeros elsewhere
• Orthonormal basis consists of mutually orthogonal unit vectors spanning the space
  • Orthogonality means the vectors are perpendicular (dot product equals zero)
  • Unit vectors have a magnitude of 1
• Rank of a matrix is the dimension of its column space (number of linearly independent columns)
  • Also equals the dimension of its row space

Coordinate Systems and Change of Basis

• Coordinate vector represents a vector in terms of a specific basis by listing the coefficients in the linear combination
  • Coordinates depend on the choice of basis
• Change of basis matrix transforms coordinates from one basis to another
  • Obtained by expressing each vector in the new basis as a linear combination of the old basis vectors
  • Invertible square matrix with columns being the new basis vectors expressed in the old basis
• Orthonormal bases simplify calculations due to the orthogonality and unit length of basis vectors
  • Dot product of basis vectors equals 0 (orthogonal) or 1 (same vector)
• Gram-Schmidt process constructs an orthonormal basis from a linearly independent set of vectors
  • Iteratively subtracts projections and normalizes the resulting vectors

Applications and Examples

• Vector spaces have numerous applications in physics, engineering, computer science, and other fields
• Euclidean spaces $\mathbb{R}^n$ are the most common examples of vector spaces
  • Vectors represent quantities with magnitude and direction (force, velocity, position)
• Function spaces contain functions as vectors with pointwise addition and scalar multiplication
  • Example: space of polynomials of degree at most $n$ with basis $\{1, x, x^2, ..., x^n\}$
• Matrix spaces consider matrices as vectors with matrix addition and scalar multiplication
  • Example: space of $m \times n$ matrices with basis being matrices with a single 1 entry and zeros elsewhere
• Quantum mechanics uses complex vector spaces (Hilbert spaces) to describe quantum states and operators
  • Basis vectors represent possible outcomes of measurements
• Fourier analysis decomposes functions into linear combinations of sinusoidal basis functions
  • Allows for efficient signal processing and data compression

Common Pitfalls and Tips

• Ensure that all vector space axioms are satisfied when determining if a set with operations forms a vector space
• Remember that subspaces must contain the zero vector and be closed under vector addition and scalar multiplication
• Be careful not to confuse linear independence with spanning; a set can be linearly independent without spanning the entire space
• When working with coordinates and change of basis, keep track of which basis is being used
• Take advantage of orthonormal bases when possible to simplify calculations and geometric interpretations
• In proofs involving linear independence or spanning, consider the zero vector and its unique representation
• Practice identifying bases, dimensions, and coordinates in various vector spaces to develop intuition
• Explore applications of vector spaces in your field of interest to deepen understanding and motivation