Linear Algebra 101 Unit 1 ReviewMatrix Algebra

Key Concepts and Definitions

• Matrices are rectangular arrays of numbers, symbols, or expressions arranged in rows and columns
  • Denoted using capital letters (A, B, C)
  • Elements of a matrix are identified by their row and column indices ($a_{ij}$ represents the element in the $i$-th row and $j$-th column)
• Vectors are special cases of matrices with only one column or one row
  • Column vectors are matrices with a single column
  • Row vectors are matrices with a single row
• Matrix dimensions refer to the number of rows and columns in a matrix
  • An $m \times n$ matrix has $m$ rows and $n$ columns
• Square matrices have an equal number of rows and columns ($n \times n$)
• Identity matrix is a square matrix with 1s on the main diagonal and 0s elsewhere
  • Denoted as $I_n$ for an $n \times n$ matrix
• Transpose of a matrix $A$, denoted as $A^T$, is obtained by interchanging the rows and columns of $A$
• Symmetric matrices are equal to their transpose ($A = A^T$)

Matrix Operations and Properties

• Matrix addition is performed element-wise and requires matrices to have the same dimensions
  • $A + B = C$, where $c_{ij} = a_{ij} + b_{ij}$
• Matrix subtraction is also performed element-wise and requires matrices to have the same dimensions
  • $A - B = C$, where $c_{ij} = a_{ij} - b_{ij}$
• Scalar multiplication involves multiplying each element of a matrix by a scalar (a single number)
  • $kA = C$, where $c_{ij} = ka_{ij}$
• Matrix multiplication is a binary operation that produces a matrix from two matrices
  • For matrices $A$ ($m \times n$) and $B$ ($n \times p$), the product $AB$ is an $m \times p$ matrix
  • Element $c_{ij}$ of the product matrix is obtained by multiplying the $i$-th row of $A$ with the $j$-th column of $B$ and summing the results
• Matrix multiplication is associative: $(AB)C = A(BC)$
• Matrix multiplication is distributive over addition: $A(B + C) = AB + AC$
• In general, matrix multiplication is not commutative: $AB \neq BA$

Systems of Linear Equations

• A system of linear equations is a collection of one or more linear equations involving the same variables
  • Example: $2x + 3y = 5$ and $x - y = 1$
• Matrices can be used to represent and solve systems of linear equations
• Augmented matrix is a matrix obtained by appending the constant terms of a system of linear equations to the coefficient matrix
  • For the system $2x + 3y = 5$ and $x - y = 1$, the augmented matrix is $\begin{bmatrix} 2 & 3 & 5 \\ 1 & -1 & 1 \end{bmatrix}$
• Gaussian elimination is a method for solving systems of linear equations by transforming the augmented matrix into row echelon form
  • Involves elementary row operations: row switching, row multiplication, and row addition
• Consistent systems have at least one solution, while inconsistent systems have no solutions
• Homogeneous systems always have the trivial solution (all variables equal to zero)

Determinants and Inverses

• The determinant of a square matrix $A$, denoted as $\det(A)$ or $|A|$, is a scalar value that provides information about the matrix's properties
  • For a $2 \times 2$ matrix $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$, $\det(A) = ad - bc$
• Determinants can be calculated using cofactor expansion or Laplace expansion for larger matrices
• A matrix is invertible (or nonsingular) if its determinant is non-zero
• The inverse of a square matrix $A$, denoted as $A^{-1}$, is a matrix such that $AA^{-1} = A^{-1}A = I$
  • Not all matrices have inverses; those without inverses are called singular matrices
• Inverses can be found using the adjugate matrix and determinant: $A^{-1} = \frac{1}{\det(A)}\adj(A)$
• Cramer's rule is a method for solving systems of linear equations using determinants

Vector Spaces and Subspaces

• A vector space is a set $V$ of objects (vectors) that satisfies certain axioms under addition and scalar multiplication
  • Closure, associativity, commutativity, identity element, inverse elements, and distributivity
• Examples of vector spaces include $\mathbb{R}^n$, the set of all $n$-tuples of real numbers, and the set of all polynomials with real coefficients
• A subspace is a subset of a vector space that is itself a vector space under the same operations
  • Must contain the zero vector and be closed under addition and scalar multiplication
• The span of a set of vectors is the set of all linear combinations of those vectors
  • A set of vectors is linearly independent if no vector in the set can be expressed as a linear combination of the others
• A basis is a linearly independent set of vectors that spans the entire vector space
  • The dimension of a vector space is the number of vectors in its basis

Linear Transformations

• A linear transformation (or linear map) is a function $T: V \to W$ between two vector spaces that preserves vector addition and scalar multiplication
  • $T(u + v) = T(u) + T(v)$ for all $u, v \in V$
  • $T(cu) = cT(u)$ for all $u \in V$ and scalars $c$
• Linear transformations can be represented by matrices
  • If $T: \mathbb{R}^n \to \mathbb{R}^m$ is a linear transformation, there exists an $m \times n$ matrix $A$ such that $T(x) = Ax$ for all $x \in \mathbb{R}^n$
• The kernel (or null space) of a linear transformation $T$ is the set of all vectors $x$ such that $T(x) = 0$
• The range (or image) of a linear transformation $T$ is the set of all vectors $y$ such that $y = T(x)$ for some $x$
• A linear transformation is injective (one-to-one) if its kernel contains only the zero vector
• A linear transformation is surjective (onto) if its range is equal to the codomain

Eigenvalues and Eigenvectors

• An eigenvector of a square matrix $A$ is a non-zero vector $v$ such that $Av = \lambda v$ for some scalar $\lambda$
  • The scalar $\lambda$ is called the eigenvalue corresponding to the eigenvector $v$
• Eigenvalues can be found by solving the characteristic equation: $\det(A - \lambda I) = 0$
• Eigenvectors can be found by solving the equation $(A - \lambda I)v = 0$ for each eigenvalue $\lambda$
• A matrix is diagonalizable if it can be written as $A = PDP^{-1}$, where $D$ is a diagonal matrix containing the eigenvalues and $P$ is a matrix whose columns are the corresponding eigenvectors
• Eigenvalues and eigenvectors have applications in physics, engineering, and computer science
  • Stability analysis, vibration modes, principal component analysis, and more

Applications and Problem Solving

• Matrices and linear algebra have numerous applications across various fields
• Markov chains use stochastic matrices to model systems that transition between states
  • Probability vectors and steady-state distributions can be found using eigenvalues and eigenvectors
• Leontief input-output models in economics use matrices to analyze the interdependencies between industries in an economy
• Computer graphics and 3D modeling heavily rely on matrices for transformations (scaling, rotation, translation, projection)
• Cryptography uses matrices in various encryption algorithms
  • Hill cipher uses matrix multiplication to encrypt and decrypt messages
• Least squares fitting and regression analysis in statistics use matrices to find the best-fitting model for a given dataset
• Fourier analysis and signal processing use matrices to represent and manipulate signals
  • Discrete Fourier transform can be expressed as a matrix multiplication
• Quantum mechanics heavily relies on linear algebra, with quantum states represented as vectors in a Hilbert space and observables as linear operators (matrices)