8.4 Vector spaces

Definition of vector spaces

A vector space is a collection of objects (called vectors) that you can add together and multiply by scalars, with the results always staying in the collection. This structure shows up everywhere in mathematics, physics, and computer science because it captures the essence of "linear" behavior in a single, clean framework.

What makes vector spaces powerful is their generality. Once you prove something about vector spaces in the abstract, that result applies to arrows in 3D space, to polynomials, to matrices, and to functions. You learn one set of rules and unlock tools for dozens of different settings.

Properties of vector spaces

A vector space over a field $\mathbb{F}$ (usually $\mathbb{R}$ or $\mathbb{C}$) must satisfy all of the following axioms. Missing even one means you don't have a vector space.

Addition axioms:

• Closure under addition: If $\vec{u}$ and $\vec{v}$ are in the space, then $\vec{u} + \vec{v}$ is also in the space.
• Associativity: $(\vec{u} + \vec{v}) + \vec{w} = \vec{u} + (\vec{v} + \vec{w})$
• Commutativity: $\vec{u} + \vec{v} = \vec{v} + \vec{u}$
• Zero vector: There exists a vector $\vec{0}$ such that $\vec{v} + \vec{0} = \vec{v}$ for all $\vec{v}$.
• Additive inverses: For every $\vec{v}$, there exists $-\vec{v}$ such that $\vec{v} + (-\vec{v}) = \vec{0}$.

Scalar multiplication axioms:

• Closure under scalar multiplication: If $c \in \mathbb{F}$ and $\vec{v}$ is in the space, then $c\vec{v}$ is in the space.
• Distributivity over vector addition: $c(\vec{u} + \vec{v}) = c\vec{u} + c\vec{v}$
• Distributivity over scalar addition: $(a + b)\vec{v} = a\vec{v} + b\vec{v}$
• Associativity of scalar multiplication: $(ab)\vec{v} = a(b\vec{v})$
• Multiplicative identity: $1\vec{v} = \vec{v}$

Examples of vector spaces

• $\mathbb{R}^n$ (real coordinate spaces): Vectors with $n$ real-number components. $\mathbb{R}^2$ is the familiar plane, $\mathbb{R}^3$ is 3D space.
• $\mathbb{C}^n$ (complex coordinate spaces): Same idea, but components are complex numbers.
• Polynomial spaces $P_n$: All polynomials of degree $\leq n$. For example, $P_2$ includes vectors like $3x^2 - x + 7$. Addition and scalar multiplication work the way you'd expect.
• Matrix spaces $M_{m \times n}$: All $m \times n$ matrices. You add them entry-by-entry and scale them entry-by-entry.
• Function spaces: Sets of functions (say, all continuous functions on $[0,1]$) that satisfy the axioms when you define addition and scaling pointwise.

Non-examples of vector spaces

Non-examples sharpen your understanding of why each axiom matters.

• The set of positive real numbers (under usual addition): The additive inverse of $5$ is $-5$, which isn't positive. No additive inverses means no vector space.
• The set of integers $\mathbb{Z}$ (as a space over $\mathbb{R}$): Scalar multiplication fails closure. Multiplying the integer $3$ by the scalar $\frac{1}{2}$ gives $1.5$, which isn't an integer.
• A circle in $\mathbb{R}^2$: Scaling a point on a unit circle by $2$ moves it off the circle. No closure under scalar multiplication.
• A plane in $\mathbb{R}^3$ that doesn't pass through the origin: It won't contain the zero vector, so it fails that axiom. (A plane through the origin, however, is a valid subspace.)

Vector operations

The two core operations in any vector space are vector addition and scalar multiplication. Everything else builds on these.

Vector addition

Vector addition combines two vectors component by component. In $\mathbb{R}^3$, for instance: $\langle 1, 2, 3 \rangle + \langle 4, -1, 0 \rangle = \langle 5, 1, 3 \rangle$.

Geometrically, you can visualize this with the parallelogram law: place the tails of two vectors at the same point, complete the parallelogram, and the diagonal is the sum.

The key properties carry over from the axioms:

• Commutative: $\vec{a} + \vec{b} = \vec{b} + \vec{a}$
• Associative: $(\vec{a} + \vec{b}) + \vec{c} = \vec{a} + (\vec{b} + \vec{c})$
• Identity: $\vec{a} + \vec{0} = \vec{a}$

Scalar multiplication

Scalar multiplication scales a vector's magnitude and can reverse its direction. Multiplying each component by the scalar does the job: $3 \langle 1, -2 \rangle = \langle 3, -6 \rangle$.

• A scalar $> 1$ stretches the vector; a scalar between $0$ and $1$ shrinks it.
• A negative scalar flips the vector's direction.
• The scalar $1$ leaves the vector unchanged: $1\vec{v} = \vec{v}$.
• Distributive properties connect the two operations: $c(\vec{a} + \vec{b}) = c\vec{a} + c\vec{b}$ and $(a + b)\vec{v} = a\vec{v} + b\vec{v}$.

Linear combinations

A linear combination takes a set of vectors, scales each one, and adds the results:

$c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_n\vec{v}_n$

The scalars $c_1, c_2, \ldots, c_n$ are called coefficients and can be any elements of the field.

Linear combinations are the central building block for nearly everything that follows. Span, linear independence, bases, and solutions to linear systems are all defined in terms of linear combinations.

Subspaces

A subspace is a subset of a vector space that is itself a vector space under the same addition and scalar multiplication. Think of it as a "smaller world" living inside a bigger one that still follows all the same rules.

Definition of subspaces

A subset $W$ of a vector space $V$ is a subspace if:

• $W$ uses the same addition and scalar multiplication as $V$.
• $W$ satisfies all the vector space axioms on its own.

Most of the axioms (associativity, commutativity, distributivity, etc.) are automatically inherited from the parent space. So you don't need to check all ten axioms from scratch. You just need the subspace test.

Criteria for subspaces (the subspace test)

To verify that $W \subseteq V$ is a subspace, check three things:

1. Non-empty: $W$ contains the zero vector $\vec{0}$.
2. Closed under addition: If $\vec{u}, \vec{v} \in W$, then $\vec{u} + \vec{v} \in W$.
3. Closed under scalar multiplication: If $\vec{v} \in W$ and $c \in \mathbb{F}$, then $c\vec{v} \in W$.

You can combine steps 2 and 3 into a single check: verify closure under linear combinations. If $\vec{u}, \vec{v} \in W$ and $a, b \in \mathbb{F}$, then $a\vec{u} + b\vec{v} \in W$.

Common subspaces

• Null space (kernel) of a matrix $A$: all vectors $\vec{x}$ such that $A\vec{x} = \vec{0}$. This is always a subspace.
• Column space (image) of a matrix: the span of the columns of $A$. Tells you which outputs are reachable.
• Row space: the span of the rows of $A$.
• Eigenspaces: for a given eigenvalue $\lambda$, the set of all vectors satisfying $A\vec{v} = \lambda\vec{v}$.
• Solution sets of homogeneous systems ($A\vec{x} = \vec{0}$) are subspaces. Non-homogeneous solution sets ($A\vec{x} = \vec{b}$ with $\vec{b} \neq \vec{0}$) are not subspaces because they don't contain $\vec{0}$.

Span and linear independence

These two concepts work together to describe the "reach" and "efficiency" of a set of vectors.

Span of vectors

The span of a set of vectors is the collection of all linear combinations you can form from them:

$\text{Span}\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n\} = \{c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_n\vec{v}_n \mid c_i \in \mathbb{F}\}$

The span is always a subspace (it's the smallest subspace containing those vectors). If the span equals the entire vector space $V$, you say the vectors span $V$, meaning every vector in $V$ can be built from them.

Linear independence vs. dependence

A set of vectors $\{\vec{v}_1, \ldots, \vec{v}_n\}$ is linearly independent if the only way to get the zero vector from a linear combination is the trivial way:

$c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_n\vec{v}_n = \vec{0} \implies c_1 = c_2 = \cdots = c_n = 0$

If there's a non-trivial combination that gives $\vec{0}$, the set is linearly dependent. That means at least one vector in the set is redundant: it can be written as a linear combination of the others.

Geometric intuition:

• In $\mathbb{R}^2$: two vectors are independent if they point in different (non-parallel) directions.
• In $\mathbb{R}^3$: three vectors are independent if they don't all lie in the same plane.

Basis of a vector space

A basis is a set of vectors that is both linearly independent and spans the entire space. It's the "just right" set: no redundant vectors, but enough to reach everything.

Key facts about bases:

• Every vector in the space can be written as a unique linear combination of basis vectors.
• A basis is a minimal spanning set (remove any vector and it no longer spans).
• A basis is a maximal independent set (add any vector and it becomes dependent).
• The standard basis for $\mathbb{R}^3$ is $\{\langle 1,0,0 \rangle, \langle 0,1,0 \rangle, \langle 0,0,1 \rangle\}$.
• Different bases give different coordinate representations of the same vectors, which is useful for simplifying problems.

Dimension of vector spaces

The dimension of a vector space is the number of vectors in any basis. This single number captures a lot about the space's structure.

Finite vs. infinite dimensions

• Finite-dimensional: $\mathbb{R}^n$ has dimension $n$. The polynomial space $P_3$ (polynomials of degree $\leq 3$) has dimension 4, because a basis is $\{1, x, x^2, x^3\}$.
• Infinite-dimensional: The space of all polynomials (no degree bound) is infinite-dimensional. So is the space of continuous functions on $[0,1]$. No finite set of vectors can span these spaces.

Dimension theorem

Every basis of a given vector space has the same number of elements. This is why dimension is well-defined: it doesn't depend on which basis you choose.

Useful consequences:

• If $\dim(V) = n$, then any set of more than $n$ vectors in $V$ must be linearly dependent.
• Any linearly independent set of exactly $n$ vectors in $V$ automatically spans $V$ (and is therefore a basis).
• If $W$ is a subspace of $V$, then $\dim(W) \leq \dim(V)$.

Coordinate systems

Once you fix a basis $\{\vec{b}_1, \ldots, \vec{b}_n\}$ for a space, every vector $\vec{v}$ can be written uniquely as:

$\vec{v} = c_1\vec{b}_1 + c_2\vec{b}_2 + \cdots + c_n\vec{b}_n$

The scalars $(c_1, c_2, \ldots, c_n)$ are the coordinates of $\vec{v}$ relative to that basis. The standard basis in $\mathbb{R}^n$ gives the familiar coordinates you're used to. Switching to a different basis can simplify a problem dramatically, which is why change of basis formulas matter.

Vector space transformations

A transformation between vector spaces is a function that sends vectors from one space to another. The most important kind preserves the linear structure.

Linear transformations

A function $T: V \to W$ is a linear transformation if it satisfies two conditions for all vectors $\vec{u}, \vec{v} \in V$ and all scalars $c$:

1. $T(\vec{u} + \vec{v}) = T(\vec{u}) + T(\vec{v})$
2. $T(c\vec{v}) = cT(\vec{v})$

These two conditions together mean $T$ preserves linear combinations. In finite-dimensional spaces, every linear transformation can be represented by a matrix. Familiar geometric operations like rotations, reflections, projections, and scaling are all linear transformations.

Kernel and image

Every linear transformation $T: V \to W$ has two important subspaces associated with it:

• Kernel (null space): $\ker(T) = \{\vec{v} \in V \mid T(\vec{v}) = \vec{0}\}$. This is a subspace of $V$. If the kernel contains only $\vec{0}$, then $T$ is injective (one-to-one).
• Image (range): $\text{im}(T) = \{T(\vec{v}) \mid \vec{v} \in V\}$. This is a subspace of $W$. If the image equals all of $W$, then $T$ is surjective (onto).

The Rank-Nullity Theorem ties these together:

$\dim(\ker(T)) + \dim(\text{im}(T)) = \dim(V)$

This is one of the most useful results in linear algebra. It tells you that the "information lost" by $T$ (the kernel) plus the "information preserved" (the image) always adds up to the dimension of the domain.

Isomorphisms between spaces

An isomorphism is a linear transformation that is both injective and surjective (bijective). Two vector spaces are isomorphic if there exists an isomorphism between them.

Isomorphic spaces are structurally identical from a linear algebra perspective. The key result: two finite-dimensional vector spaces over the same field are isomorphic if and only if they have the same dimension. So $\mathbb{R}^3$, $P_2$, and $M_{1 \times 3}$ are all isomorphic because each has dimension 3.

Inner product spaces

An inner product space is a vector space equipped with an additional operation called an inner product. This operation lets you define geometric concepts like length, distance, and angle within the algebraic framework of vector spaces.

Definition of inner products

An inner product on a vector space $V$ is a function $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{F}$ satisfying:

• Positive definiteness: $\langle \vec{v}, \vec{v} \rangle \geq 0$, with equality only when $\vec{v} = \vec{0}$.
• Conjugate symmetry: $\langle \vec{u}, \vec{v} \rangle = \overline{\langle \vec{v}, \vec{u} \rangle}$. (For real spaces, this simplifies to plain symmetry: $\langle \vec{u}, \vec{v} \rangle = \langle \vec{v}, \vec{u} \rangle$.)
• Linearity in the first argument: $\langle a\vec{u} + b\vec{v}, \vec{w} \rangle = a\langle \vec{u}, \vec{w} \rangle + b\langle \vec{v}, \vec{w} \rangle$.

The standard example is the dot product in $\mathbb{R}^n$: $\langle \vec{u}, \vec{v} \rangle = u_1v_1 + u_2v_2 + \cdots + u_nv_n$.

From an inner product, you can define the norm (length) of a vector: $\|\vec{v}\| = \sqrt{\langle \vec{v}, \vec{v} \rangle}$.

Orthogonality and orthonormality

Two vectors are orthogonal if $\langle \vec{u}, \vec{v} \rangle = 0$. In geometric terms, they're perpendicular.

An orthonormal set goes one step further: the vectors are orthogonal and each has unit length ($\|\vec{v}\| = 1$).

Why care? Orthonormal bases make computations much cleaner. If $\{\vec{e}_1, \ldots, \vec{e}_n\}$ is an orthonormal basis, then the coordinates of any vector $\vec{v}$ are simply $c_i = \langle \vec{v}, \vec{e}_i \rangle$. No systems of equations needed.

Gram-Schmidt process

The Gram-Schmidt process takes any linearly independent set and produces an orthonormal basis spanning the same subspace.

Steps:

1. Start with a linearly independent set $\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n\}$.

2. Set $\vec{u}_1 = \vec{v}_1$.

3. For each subsequent vector $\vec{v}_k$, subtract off its projections onto all previously computed $\vec{u}$'s: $\vec{u}_k = \vec{v}_k - \sum_{j=1}^{k-1} \frac{\langle \vec{v}_k, \vec{u}_j \rangle}{\langle \vec{u}_j, \vec{u}_j \rangle} \vec{u}_j$

4. Normalize each $\vec{u}_k$ to get a unit vector: $\vec{e}_k = \frac{\vec{u}_k}{\|\vec{u}_k\|}$.

The result $\{\vec{e}_1, \ldots, \vec{e}_n\}$ is orthonormal and spans the same subspace as the original set. This process is used in least squares problems, QR factorization, and quantum mechanics.

Applications of vector spaces

Vector spaces aren't just abstract theory. They provide the language and tools for solving concrete problems across many fields.

Linear algebra connections

• Systems of linear equations translate directly into questions about spans, null spaces, and column spaces.
• Eigenvalue problems arise in differential equations and dynamical systems, where you need to find vectors that a transformation only scales.
• Matrix decompositions (SVD, LU, QR) break matrices into simpler pieces for data compression, numerical stability, and analysis.
• Least squares approximation finds the "best fit" when an exact solution doesn't exist, using projections onto subspaces.
• Fourier analysis represents signals as linear combinations of sine and cosine functions, treating them as vectors in a function space.

Physics and engineering uses

• Quantum mechanics represents particle states as vectors in Hilbert spaces (infinite-dimensional inner product spaces).
• Electromagnetic theory models fields as vector-valued functions across space.
• Structural engineering uses finite element methods, which discretize continuous structures into systems of linear equations.
• Control theory describes dynamic systems using state vectors and linear transformations.
• Robotics relies on transformation matrices for motion planning and positioning.

Computer graphics applications

• 3D transformations (rotation, translation, scaling) manipulate objects and cameras using matrix multiplication.
• Texture mapping applies coordinate transformations to wrap images onto surfaces.
• Ray tracing computes light paths using vector operations to produce realistic images.
• Animation interpolates between positions and orientations in vector spaces to create smooth motion.
• Color spaces treat colors as vectors, and transformations convert between different color representations (RGB, HSV, etc.).