2.2 Vector Spaces and Linear Independence

Vector spaces and linear independence are key concepts in linear algebra. They provide a foundation for understanding the structure of mathematical systems used in coding theory.

These concepts help us analyze and manipulate data in multiple dimensions. By grasping vector spaces and linear independence, we can better understand error-correcting codes and their properties.

Vector Spaces and Subspaces

Definition and Properties of Vector Spaces

• Vector space consists of a set $V$ of vectors and two operations (vector addition and scalar multiplication) that satisfy certain axioms
  • Closure under vector addition: Adding any two vectors in $V$ results in another vector in $V$
  • Closure under scalar multiplication: Multiplying any vector in $V$ by a scalar (real or complex number) results in another vector in $V$
  • Associativity of vector addition: $(u + v) + w = u + (v + w)$ for all vectors $u$, $v$, and $w$ in $V$
  • Commutativity of vector addition: $u + v = v + u$ for all vectors $u$ and $v$ in $V$
  • Existence of additive identity: There exists a unique vector $0$ (zero vector) such that $v + 0 = v$ for all vectors $v$ in $V$
  • Existence of additive inverses: For every vector $v$ in $V$, there exists a unique vector $-v$ such that $v + (-v) = 0$
• Examples of vector spaces include
  • $\mathbb{R}^n$: The set of all $n$-tuples of real numbers $(x_1, x_2, \ldots, x_n)$
  • $\mathbb{C}^n$: The set of all $n$-tuples of complex numbers $(z_1, z_2, \ldots, z_n)$
  • $\mathcal{P}_n$: The set of all polynomials of degree at most $n$

Subspaces and Their Properties

• Subspace is a non-empty subset $W$ of a vector space $V$ that is itself a vector space under the same operations as $V$
  • Closure under vector addition: If $u$ and $v$ are in $W$, then $u + v$ is also in $W$
  • Closure under scalar multiplication: If $v$ is in $W$ and $c$ is a scalar, then $cv$ is also in $W$
• Examples of subspaces include
  • The zero vector space $\{0\}$ is a subspace of any vector space
  • The set of all polynomials of degree at most $k$ is a subspace of $\mathcal{P}_n$ for $k \leq n$
• To prove a subset is a subspace, show it satisfies the subspace properties or use the subspace test
  • Subspace test: A non-empty subset $W$ of a vector space $V$ is a subspace if and only if for any $u, v \in W$ and any scalar $c$, we have $u + v \in W$ and $cu \in W$

Linear Combinations and Span

• Linear combination of vectors $v_1, v_2, \ldots, v_k$ in a vector space $V$ is a vector of the form $c_1v_1 + c_2v_2 + \cdots + c_kv_k$, where $c_1, c_2, \ldots, c_k$ are scalars
  • Coefficients $c_1, c_2, \ldots, c_k$ can be any scalars (real or complex numbers)
  • Example: In $\mathbb{R}^3$, if $v_1 = (1, 0, 0)$, $v_2 = (0, 1, 0)$, and $v_3 = (0, 0, 1)$, then $(2, -3, 5)$ is a linear combination of $v_1$, $v_2$, and $v_3$ with coefficients $c_1 = 2$, $c_2 = -3$, and $c_3 = 5$
• Span of a set of vectors $\{v_1, v_2, \ldots, v_k\}$ in a vector space $V$ is the set of all linear combinations of these vectors
  • Denoted as $\text{span}(v_1, v_2, \ldots, v_k)$ or $\langle v_1, v_2, \ldots, v_k \rangle$
  • Span is always a subspace of $V$
  • Example: In $\mathbb{R}^3$, the span of $v_1 = (1, 0, 0)$ and $v_2 = (0, 1, 0)$ is the $xy$-plane

Basis and Dimension

Basis of a Vector Space

• Basis of a vector space $V$ is a linearly independent set of vectors that spans $V$
  • Linearly independent: No vector in the set can be written as a linear combination of the other vectors
  • Spans $V$: Every vector in $V$ can be written as a linear combination of the basis vectors
• Examples of bases include
  • Standard basis for $\mathbb{R}^n$: $\{e_1, e_2, \ldots, e_n\}$, where $e_i$ has a 1 in the $i$-th position and 0s elsewhere
  • Basis for $\mathcal{P}_2$: $\{1, x, x^2\}$
• Every vector space has a basis, and any two bases of a vector space have the same number of elements

Dimension of a Vector Space

• Dimension of a vector space $V$ is the number of vectors in any basis of $V$
  • Denoted as $\dim(V)$
  • All bases of a vector space have the same number of elements, so the dimension is well-defined
• Examples of dimensions include
  • $\dim(\mathbb{R}^n) = n$
  • $\dim(\mathcal{P}_n) = n + 1$
• Dimension provides a measure of the "size" of a vector space
  • Finite-dimensional vector space: A vector space with a finite basis (and thus a finite dimension)
  • Infinite-dimensional vector space: A vector space that is not finite-dimensional

Coordinate Vectors

• Coordinate vector of a vector $v$ with respect to a basis $\{v_1, v_2, \ldots, v_n\}$ is the unique $n$-tuple $(c_1, c_2, \ldots, c_n)$ such that $v = c_1v_1 + c_2v_2 + \cdots + c_nv_n$
  • Represents $v$ as a linear combination of the basis vectors
  • Uniqueness follows from the linear independence of the basis vectors
• Example: In $\mathbb{R}^3$ with the standard basis $\{e_1, e_2, e_3\}$, the coordinate vector of $(2, -3, 5)$ is $(2, -3, 5)$
• Coordinate vectors provide a way to represent vectors in terms of a basis
  • Allows for computation and analysis using the coordinates rather than the vectors themselves
  • Coordinate vectors depend on the chosen basis

Linear Independence

Definition and Properties of Linear Independence

• Set of vectors $\{v_1, v_2, \ldots, v_k\}$ in a vector space $V$ is linearly independent if the equation $c_1v_1 + c_2v_2 + \cdots + c_kv_k = 0$ has only the trivial solution $c_1 = c_2 = \cdots = c_k = 0$
  • Equivalent to saying that no vector in the set can be written as a linear combination of the other vectors
  • Example: In $\mathbb{R}^3$, the vectors $(1, 0, 0)$, $(0, 1, 0)$, and $(0, 0, 1)$ are linearly independent
• Set of vectors is linearly dependent if it is not linearly independent
  • Equivalent to saying that at least one vector in the set can be written as a linear combination of the other vectors
  • Example: In $\mathbb{R}^3$, the vectors $(1, 0, 0)$, $(0, 1, 0)$, and $(1, 1, 0)$ are linearly dependent, as $(1, 1, 0) = (1, 0, 0) + (0, 1, 0)$
• Properties of linearly independent sets
  • Any subset of a linearly independent set is linearly independent
  • If a set contains the zero vector, it is linearly dependent
  • In a vector space of dimension $n$, any set of more than $n$ vectors is linearly dependent

Determining Linear Independence

• To determine if a set of vectors $\{v_1, v_2, \ldots, v_k\}$ is linearly independent, solve the equation $c_1v_1 + c_2v_2 + \cdots + c_kv_k = 0$ for the coefficients $c_1, c_2, \ldots, c_k$
  • If the only solution is the trivial solution $(c_1, c_2, \ldots, c_k) = (0, 0, \ldots, 0)$, the set is linearly independent
  • If there are non-trivial solutions, the set is linearly dependent
• Example: To determine if the vectors $(1, 2, 3)$, $(2, 1, 1)$, and $(3, 5, 7)$ in $\mathbb{R}^3$ are linearly independent, solve the equation $c_1(1, 2, 3) + c_2(2, 1, 1) + c_3(3, 5, 7) = (0, 0, 0)$
  • This leads to a system of linear equations, which can be solved using techniques like Gaussian elimination
  • In this case, the only solution is $(c_1, c_2, c_3) = (0, 0, 0)$, so the vectors are linearly independent

Importance of Linear Independence in Bases

• Linear independence is a crucial property for bases of vector spaces
  • Ensures that each vector in the basis is not redundant and contributes to spanning the entire vector space
  • Guarantees that every vector in the vector space has a unique representation as a linear combination of the basis vectors
• In a finite-dimensional vector space, a set of vectors is a basis if and only if it is linearly independent and spans the vector space
  • Linear independence ensures that the set is not "too large"
  • Spanning ensures that the set is "large enough" to generate all vectors in the space
• Example: In $\mathbb{R}^3$, the standard basis $\{(1, 0, 0), (0, 1, 0), (0, 0, 1)\}$ is linearly independent and spans $\mathbb{R}^3$, making it a basis for the vector space