Key Concepts of Systems of Linear Equations

---

Why This Matters

Systems of linear equations form the backbone of linear algebra. They're the framework you'll use to analyze everything from circuit networks to economic models to computer graphics transformations. When you encounter these systems, you're really being tested on your ability to recognize solution structures, apply systematic elimination techniques, and interpret what the algebra tells you about the underlying geometry. The concepts here (consistency, rank, independence) will resurface constantly throughout the course.

Don't just memorize definitions and procedures. For each concept below, understand what it reveals about the system's behavior and when you'd choose one method over another. Exam questions love to ask you to identify solution types from matrix forms, explain why a particular method works, or connect algebraic results to geometric interpretations.

---

Foundations: Setting Up the System

Before you can solve anything, you need to understand how systems are structured and represented. These foundational concepts establish the language we use to describe and manipulate linear systems.

Definition of a System of Linear Equations

A system of linear equations is a collection of linear equations that share the same set of variables. Each equation represents a hyperplane in $n$-dimensional space (for two variables, that's just a line; for three, a plane).

The solution set consists of all variable values that satisfy every equation at the same time. Geometrically, you're looking for where all those hyperplanes intersect. The key constraint is that variables appear only to the first power, with no products between them. This linearity is exactly what makes systematic solution methods possible.

Coefficient Matrix and Augmented Matrix

• The coefficient matrix $A$ contains only the variable coefficients, with each equation as a row. You'll use it when analyzing system properties like rank or computing determinants.
• The augmented matrix $[A \mid b]$ appends the constant terms after a vertical line. This is your working matrix for elimination methods.
• Row operations on the augmented matrix preserve the solution set while transforming the system into easier-to-solve forms.

Vector Form of Linear Systems

You can express any system as a single matrix equation: $A\mathbf{x} = \mathbf{b}$. This replaces multiple scalar equations with one compact expression.

The geometric interpretation is powerful: solving $A\mathbf{x} = \mathbf{b}$ means finding a linear combination of the column vectors of $A$ that produces $\mathbf{b}$. This viewpoint connects directly to the concept of column space, which you'll encounter later in the course.

---

Classifying Solutions: What Can Happen?

Every system falls into one of three categories. Understanding why each occurs, and how to identify which you're dealing with, is fundamental to everything that follows.

Consistent and Inconsistent Systems

A consistent system has at least one solution. The equations' geometric representations share at least one common point.

An inconsistent system has no solutions. In row echelon form, you'll spot this as a row like $[0 \; 0 \; \cdots \; 0 \; | \; c]$ where $c \neq 0$. That row translates to $0 = c$, which is a contradiction.

The test for consistency comes down to rank: if $\text{rank}(A) = \text{rank}([A \mid b])$, the system is consistent. If the augmented matrix has higher rank, the extra pivot sits in the constants column, producing a contradiction.

Unique, Infinite, and No Solutions

• Unique solution: the system has exactly as many independent equations as unknowns. Geometrically, $n$ hyperplanes intersect at a single point.
• Infinitely many solutions: some equations are redundant (dependent), so the constraints don't fully pin down the variables. The solution set forms a line, plane, or higher-dimensional subspace.
• No solution: the equations contradict each other. Think parallel lines that never meet.

Homogeneous Systems

A homogeneous system has all constant terms equal to zero: $A\mathbf{x} = \mathbf{0}$. The zero vector $\mathbf{x} = \mathbf{0}$ is always a solution (the trivial solution), so homogeneous systems are always consistent.

The real question is whether non-trivial solutions exist. They do whenever there are free variables, which happens when the number of variables exceeds $\text{rank}(A)$. For example, a homogeneous system with 4 variables and $\text{rank}(A) = 2$ has a two-dimensional solution space.

---

The Elimination Toolkit: Transforming Systems

These are your primary tools for systematically solving systems. Master the operations and the forms they produce.

Elementary Row Operations

There are exactly three permitted operations:

1. Swap two rows.
2. Scale a row by multiplying it by a non-zero scalar.
3. Replace a row by adding a scalar multiple of another row to it.

These are solution-preserving transformations: the modified system has exactly the same solutions as the original. Every matrix reduction method is just a strategic sequence of these three operations.

Gaussian Elimination

Gaussian elimination is a systematic process of forward elimination that transforms the augmented matrix into row echelon form (REF).

Steps:

1. Identify the leftmost column that has a non-zero entry. This is your pivot column.
2. If needed, swap rows to place a non-zero entry (the pivot) at the top of this column.
3. Use row replacement operations to create zeros in all entries below the pivot.
4. Move to the next row and the next pivot column. Repeat steps 1-3 on the submatrix below and to the right of the current pivot.
5. Continue until the matrix is in row echelon form.

During this process, the solution type reveals itself. A contradictory row (like $[0 \; 0 \; 0 \; | \; 5]$) means no solution. Columns without pivots correspond to free variables, indicating infinitely many solutions.

Row Echelon Form and Reduced Row Echelon Form

• Row echelon form (REF): all-zero rows sit at the bottom, each leading entry (pivot) is to the right of the pivot above it, and all entries below each pivot are zero. This creates a "staircase" pattern.
• Reduced row echelon form (RREF): satisfies everything REF does, plus each pivot is 1 and is the only non-zero entry in its column.

RREF gives solutions directly without back-substitution. Each pivot column's variable equals the corresponding entry in the constants column. Every matrix has a unique RREF, which makes it especially useful for theoretical analysis.

Back-Substitution

After reaching REF (but not RREF), you still need to extract the solution. Back-substitution works from the bottom up:

1. Solve the last equation for its leading variable.
2. Substitute that value into the equation above and solve for the next variable.
3. Continue upward until all pivot variables are determined.

If you've gone all the way to RREF, back-substitution is unnecessary since the matrix already displays the solution.

---

Measuring System Structure: Independence and Rank

These concepts tell you how much information your equations actually contain. They're crucial for predicting solution behavior before you even start solving.

Linear Independence and Dependence

Independent equations each contribute a unique constraint. None of them can be written as a combination of the others. Dependent equations contain redundancy: at least one equation is a linear combination of the rest and adds no new information.

This directly determines solution behavior. With $n$ independent equations in $n$ unknowns, you get a unique solution. If some equations are dependent, you end up with fewer real constraints than unknowns, which creates free variables and infinitely many solutions.

Rank of a Matrix

The rank of a matrix is the number of linearly independent rows (equivalently, the number of pivots in row echelon form). It tells you the actual number of independent constraints in the system.

The rank-nullity theorem connects rank to the solution space:

$\text{rank}(A) + \text{nullity}(A) = n$

where $n$ is the number of columns (variables) and $\text{nullity}(A)$ is the dimension of the null space (the number of free variables).

Solution classification using rank:

• $\text{rank}(A) = \text{rank}([A \mid b]) = n$ → unique solution
• $\text{rank}(A) = \text{rank}([A \mid b]) < n$ → infinitely many solutions
• $\text{rank}(A) < \text{rank}([A \mid b])$ → no solution (inconsistent)

Parametric Solutions

When a system has infinitely many solutions, you express them using free variables as parameters. Each free variable can take any real value.

The general solution takes the form: particular solution + any linear combination of homogeneous solutions. The number of parameters equals $n - \text{rank}(A)$, which is the dimension of the solution space.

For example, if you have 4 variables and $\text{rank}(A) = 2$, your solution will have $4 - 2 = 2$ free parameters.

---

Alternative Solution Methods

Beyond Gaussian elimination, these methods offer different advantages depending on system size and structure.

Cramer's Rule

Cramer's rule uses determinants to solve square systems ($n$ equations, $n$ unknowns) where the coefficient matrix is invertible. The formula for each variable is:

$x_i = \frac{\det(A_i)}{\det(A)}$

where $A_i$ is the matrix formed by replacing column $i$ of $A$ with the constant vector $\mathbf{b}$.

This is practical only for small systems ($2 \times 2$ or $3 \times 3$). For larger matrices, computing determinants becomes very expensive compared to Gaussian elimination.

Matrix Inverse Method

When $A$ is square and invertible ($\det(A) \neq 0$), you can solve $A\mathbf{x} = \mathbf{b}$ directly:

$\mathbf{x} = A^{-1}\mathbf{b}$

If the determinant is zero, $A$ has no inverse and this method fails. The real advantage shows up when you need to solve multiple systems with the same coefficient matrix but different constant vectors. You compute $A^{-1}$ once and then multiply by each new $\mathbf{b}$.

---

Quick Reference Table

---

Self-Check Questions

1. Given an augmented matrix in row echelon form, how do you determine whether the system is consistent, and if consistent, whether the solution is unique or infinite?

2. Compare Gaussian elimination to the matrix inverse method: what conditions must be met to use each, and when would you prefer one over the other?

3. A homogeneous system $A\mathbf{x} = \mathbf{0}$ has 5 variables and $\text{rank}(A) = 3$. How many free parameters appear in the general solution, and why is the system guaranteed to have non-trivial solutions?

4. What's the relationship between the rank of the coefficient matrix and the rank of the augmented matrix for consistent systems? How does this change for inconsistent systems?

5. If you're asked to solve a $3 \times 3$ system for only one specific variable and you know the coefficient matrix is invertible, which method would be most efficient, and why?