Key Concepts of Vector Spaces

Why This Matters

Vector spaces are the foundation for nearly everything you'll encounter in linear algebra and beyond. Differential equations, computer graphics, quantum mechanics, and data science all rely on vector spaces. The concepts here, spanning, independence, dimension, and transformations, give you the language to describe how mathematical objects behave and interact.

What you're really being tested on: can you recognize why certain vectors form a basis, how transformations preserve structure, and what properties make a subspace valid? Don't just memorize definitions. Know what each concept illustrates and how they connect. A basis is the "minimal spanning set," and dimension tells you the "degrees of freedom" in a space. Understanding those ideas will carry you through both computational problems and conceptual questions.

---

Foundational Structures

Every vector space problem starts with understanding what makes a space valid and how smaller spaces live inside larger ones. Vector spaces are defined entirely by their behavior under two operations: addition and scalar multiplication.

Definition of a Vector Space

A vector space is a set $V$ equipped with vector addition and scalar multiplication that satisfies eight axioms. These include closure under both operations, associativity and commutativity of addition, existence of an additive identity (the zero vector) and additive inverses, and compatibility of scalar multiplication with field operations (distributive laws, multiplicative identity).

• Scalars come from a field, usually $\mathbb{R}$ or $\mathbb{C}$. The field determines what "multiplication by a scalar" means.
• Examples range from concrete to abstract. $\mathbb{R}^n$ (columns of $n$ real numbers), polynomial spaces $P_n$ (polynomials of degree at most $n$), and continuous function spaces $C[a,b]$ all satisfy these axioms.

The eight axioms might feel like a lot, but most of the time you're working in spaces where they're clearly satisfied. Where they really matter is when you need to show something isn't a vector space (for instance, a set where addition doesn't stay inside the set).

Subspaces

A subspace is a subset of a vector space that is itself a vector space under the same operations. Rather than checking all eight axioms, you only need three conditions (the subspace test):

1. The subset contains the zero vector $\vec{0}$.
2. It's closed under addition: if $\vec{u}$ and $\vec{v}$ are in the subset, so is $\vec{u} + \vec{v}$.
3. It's closed under scalar multiplication: if $\vec{v}$ is in the subset and $c$ is a scalar, then $c\vec{v}$ is in the subset.

A subspace always passes through the origin. A plane in $\mathbb{R}^3$ is only a subspace if it contains $\vec{0}$. A plane that's been shifted away from the origin fails the test.

One common exam trap: the intersection of two subspaces is always a subspace, but the union generally is not. Think about two different lines through the origin in $\mathbb{R}^2$. Their union contains vectors from both lines, but adding a vector from one line to a vector from the other can land you outside both lines.

---

Independence and Spanning

These twin concepts determine whether you have "enough" vectors and whether you have "too many." Linear independence means no redundancy; span means complete coverage.

Linear Independence and Dependence

A set of vectors $\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n\}$ is linearly independent if the only solution to

$c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_n\vec{v}_n = \vec{0}$

is $c_1 = c_2 = \cdots = c_n = 0$ (the trivial solution). If any nontrivial solution exists, the set is linearly dependent, meaning at least one vector can be written as a combination of the others.

• Geometric interpretation: independent vectors point in "genuinely different" directions. Dependent vectors include at least one that's redundant.
• In $\mathbb{R}^n$, you can never have more than $n$ linearly independent vectors. This is a hard ceiling that limits basis size.

To test independence in practice, set up the equation above, form the corresponding matrix, and row reduce. If every column has a pivot, the set is independent.

Span of Vectors

The span of a set of vectors is the collection of all possible linear combinations of those vectors:

$\text{span}\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_k\} = \{c_1\vec{v}_1 + c_2\vec{v}_2 + \cdots + c_k\vec{v}_k : c_i \in \mathbb{R}\}$

• Adding more vectors to a set can only expand or maintain the span. It never shrinks it.
• The span of a single nonzero vector in $\mathbb{R}^3$ is a line through the origin. The span of two independent vectors is a plane through the origin.

Basis and Dimension

A basis for a vector space $V$ is a set of vectors that is both linearly independent and spans $V$. It's the minimal spanning set: remove any vector and you lose coverage; add any vector and you introduce redundancy.

• Dimension equals the number of vectors in any basis. For example, $\dim(\mathbb{R}^n) = n$, and $\dim(P_2) = 3$ because $\{1, x, x^2\}$ is a basis for polynomials of degree at most 2.
• All bases of a given space have the same number of vectors. This is a theorem, not obvious, and it's the reason dimension is well-defined.

---

Linear Transformations and Their Properties

Transformations are functions between vector spaces that preserve the vector space structure. The preservation of addition and scalar multiplication is exactly what makes them "linear."

Linear Transformations

A function $T: V \to W$ is a linear transformation if it satisfies two properties for all vectors $\vec{u}, \vec{v} \in V$ and all scalars $c$:

1. $T(\vec{u} + \vec{v}) = T(\vec{u}) + T(\vec{v})$
2. $T(c\vec{v}) = cT(\vec{v})$

These two conditions can be combined into one: $T(c_1\vec{u} + c_2\vec{v}) = c_1T(\vec{u}) + c_2T(\vec{v})$.

• Matrix representation: every linear transformation $T: \mathbb{R}^n \to \mathbb{R}^m$ corresponds to an $m \times n$ matrix $A$, so that $T(\vec{v}) = A\vec{v}$.
• Composition of transformations corresponds to matrix multiplication. Order matters: $T_2 \circ T_1$ means apply $T_1$ first, which corresponds to $A_2 A_1$.

Null Space and Range

• The null space (or kernel) of $T$ is $\{\vec{v} \in V : T(\vec{v}) = \vec{0}\}$. It captures what the transformation "destroys," the inputs that get mapped to zero.
• The range (or image) of $T$ is $\{T(\vec{v}) : \vec{v} \in V\}$. It captures what outputs are actually achievable.

Both the null space and the range are subspaces (of the domain and codomain, respectively).

The Rank-Nullity Theorem ties them together:

$\dim(\text{null } T) + \dim(\text{range } T) = \dim(\text{domain})$

This is your most powerful dimension-counting tool. If you know any two of these three quantities, you can find the third.

---

Eigentheory

Eigenvectors reveal the "natural directions" of a transformation: directions that get stretched or compressed but not rotated. This is where linear algebra connects directly to dynamics and differential equations.

Eigenvalues and Eigenvectors

The defining equation is:

$A\vec{v} = \lambda\vec{v}, \quad \vec{v} \neq \vec{0}$

The nonzero vector $\vec{v}$ is an eigenvector, and the scalar $\lambda$ is the corresponding eigenvalue. Under the transformation $A$, the eigenvector only gets scaled by $\lambda$.

Finding eigenvalues step by step:

1. Rewrite $A\vec{v} = \lambda\vec{v}$ as $(A - \lambda I)\vec{v} = \vec{0}$.
2. For a nonzero solution $\vec{v}$ to exist, the matrix $(A - \lambda I)$ must be singular.
3. Solve the characteristic equation: $\det(A - \lambda I) = 0$. The solutions are the eigenvalues.
4. For each eigenvalue $\lambda$, find the eigenvectors by solving $(A - \lambda I)\vec{v} = \vec{0}$ (i.e., find the null space of $A - \lambda I$).

The set of all eigenvectors for a given $\lambda$, together with the zero vector, forms the eigenspace for that eigenvalue.

Applications are everywhere: stability analysis of systems, principal component analysis in statistics, and solving systems of differential equations of the form $\vec{x}' = A\vec{x}$.

---

Inner Product Structures

Inner products add geometry to algebra. With an inner product, you can talk about lengths, angles, and perpendicularity. The inner product generalizes the familiar dot product to abstract vector spaces.

Inner Product Spaces

An inner product on a vector space $V$ is a function $\langle \cdot, \cdot \rangle$ that takes two vectors and returns a scalar, satisfying:

• Linearity in the first argument: $\langle a\vec{u} + b\vec{v}, \vec{w} \rangle = a\langle \vec{u}, \vec{w} \rangle + b\langle \vec{v}, \vec{w} \rangle$
• Symmetry (or conjugate symmetry over $\mathbb{C}$): $\langle \vec{u}, \vec{v} \rangle = \langle \vec{v}, \vec{u} \rangle$
• Positive definiteness: $\langle \vec{v}, \vec{v} \rangle > 0$ for all $\vec{v} \neq \vec{0}$, and $\langle \vec{0}, \vec{0} \rangle = 0$

The inner product induces a norm (a notion of length): $\|\vec{v}\| = \sqrt{\langle \vec{v}, \vec{v} \rangle}$.

The standard example is $\mathbb{R}^n$ with the dot product: $\langle \vec{u}, \vec{v} \rangle = \vec{u} \cdot \vec{v} = \sum_{i=1}^{n} u_i v_i$.

Orthogonality and Orthonormal Bases

Two vectors are orthogonal if $\langle \vec{u}, \vec{v} \rangle = 0$. An orthonormal set goes one step further: every vector is also a unit vector ($\|\vec{v}\| = 1$).

The Gram-Schmidt process converts any basis into an orthonormal basis. Here's the idea for two vectors $\vec{v}_1, \vec{v}_2$:

1. Set $\vec{u}_1 = \vec{v}_1$.
2. Subtract the component of $\vec{v}_2$ along $\vec{u}_1$: $\vec{u}_2 = \vec{v}_2 - \frac{\langle \vec{v}_2, \vec{u}_1 \rangle}{\langle \vec{u}_1, \vec{u}_1 \rangle}\vec{u}_1$.
3. Normalize each $\vec{u}_i$ by dividing by its norm: $\vec{e}_i = \frac{\vec{u}_i}{\|\vec{u}_i\|}$.

For more vectors, repeat step 2 by subtracting projections onto all previously computed vectors.

With an orthonormal basis $\{\vec{e}_1, \ldots, \vec{e}_n\}$, finding coordinates becomes trivial. The coefficient for each basis vector is just:

$c_i = \langle \vec{v}, \vec{e}_i \rangle$

No system of equations to solve, no matrix inversion needed.

---

Quick Reference Table

---

Self-Check Questions

1. If a set of vectors spans $\mathbb{R}^4$ but contains 5 vectors, what can you conclude about their linear independence? Why?

2. Compare and contrast the null space and range of a linear transformation. How does the Rank-Nullity Theorem connect them?

3. A subspace of $\mathbb{R}^3$ has dimension 2. What geometric object does it represent, and what must be true about its relationship to the origin?

4. Given a matrix with eigenvalue $\lambda = 0$, what can you immediately conclude about the matrix's invertibility and null space?

5. Why does an orthonormal basis make finding the coordinates of a vector so much simpler than an arbitrary basis? What formula would you use?