1.2 Explain inverse of a matrix

Matrix inverses are a crucial concept in linear algebra. They allow us to undo matrix operations and solve complex systems of equations. Understanding inverses helps us grasp how matrices transform space and how to reverse those transformations.

Inverses have unique properties that make them powerful tools. They're used in various fields, from computer graphics to economics. Knowing how to calculate and use inverses is essential for solving real-world problems involving linear systems and transformations.

Matrix Inverses and Properties

Definition and Properties of Matrix Inverses

• The inverse of a square matrix $A$, denoted as $A^{-1}$, is another square matrix such that $A * A^{-1} = A^{-1} * A = I$, where $I$ is the identity matrix
• If $A$ is an invertible matrix, then $A^{-1}$ is unique
• The inverse of a matrix, if it exists, has the same dimensions as the original matrix
• The product of a matrix and its inverse is commutative, meaning $A * A^{-1} = A^{-1} * A$
• The inverse of the inverse of a matrix $A$ is the matrix $A$ itself, i.e., $(A^{-1})^{-1} = A$

Properties of Matrix Products and Inverses

• The inverse of a product of matrices is equal to the product of their inverses in reverse order, i.e., $(AB)^{-1} = B^{-1} * A^{-1}$
  • For example, if $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$ and $B = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix}$, then $(AB)^{-1} = B^{-1} * A^{-1}$
• The inverse of a diagonal matrix is obtained by reciprocating each non-zero diagonal element
  • For example, if $D = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}$, then $D^{-1} = \begin{bmatrix} \frac{1}{2} & 0 \\ 0 & \frac{1}{3} \end{bmatrix}$
• The inverse of an orthogonal matrix is equal to its transpose, i.e., $Q^{-1} = Q^T$ for an orthogonal matrix $Q$

Conditions for Matrix Invertibility

Nonsingularity and Invertibility

• A square matrix $A$ has an inverse if and only if it is nonsingular or invertible
• A matrix is invertible if and only if its determinant is not equal to zero, i.e., $\det(A) \neq 0$
  • For example, if $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$, then $\det(A) = -2 \neq 0$, so $A$ is invertible
• A matrix is invertible if and only if its rank is equal to its dimension, i.e., $\text{rank}(A) = n$ for an $n \times n$ matrix

Linear Independence and Invertibility

• A matrix is invertible if and only if it has linearly independent columns or rows
  • For example, if $A = \begin{bmatrix} 1 & 2 \\ 3 & 6 \end{bmatrix}$, then its columns are linearly dependent (the second column is a multiple of the first), so $A$ is not invertible
• If a matrix $A$ is invertible, then the linear transformation represented by $A$ is bijective (one-to-one and onto)
• A square matrix $A$ is invertible if and only if the equation $Ax = 0$ has only the trivial solution $x = 0$

Calculating Matrix Inverses

Gaussian Elimination Method

• Gaussian elimination method: Augment the matrix $A$ with the identity matrix $I$ and perform row operations to transform $A$ into $I$. The resulting right-hand side will be the inverse of $A$
  • For example, to find the inverse of $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$, augment $A$ with $I$ to get $\begin{bmatrix} 1 & 2 & 1 & 0 \\ 3 & 4 & 0 & 1 \end{bmatrix}$, then perform row operations to obtain $\begin{bmatrix} 1 & 0 & -2 & 1 \\ 0 & 1 & \frac{3}{2} & -\frac{1}{2} \end{bmatrix}$, so $A^{-1} = \begin{bmatrix} -2 & 1 \\ \frac{3}{2} & -\frac{1}{2} \end{bmatrix}$
• This method is generally applicable to any square invertible matrix

Adjugate and Cofactor Methods

• Adjugate method (Cramer's rule): $A^{-1} = \frac{1}{\det(A)} * \text{adj}(A)$, where $\text{adj}(A)$ is the adjugate matrix of $A$, obtained by transposing the cofactor matrix of $A$
• Cofactor expansion method: Compute the cofactor matrix of $A$, transpose it, and divide each element by the determinant of $A$
  • For example, to find the inverse of $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$, first compute the cofactor matrix $C = \begin{bmatrix} 4 & -2 \\ -3 & 1 \end{bmatrix}$, then $\text{adj}(A) = C^T = \begin{bmatrix} 4 & -3 \\ -2 & 1 \end{bmatrix}$, and finally $A^{-1} = \frac{1}{\det(A)} * \text{adj}(A) = \frac{1}{-2} * \begin{bmatrix} 4 & -3 \\ -2 & 1 \end{bmatrix} = \begin{bmatrix} -2 & \frac{3}{2} \\ 1 & -\frac{1}{2} \end{bmatrix}$
• These methods are more computationally expensive than Gaussian elimination but can be useful for theoretical purposes

Other Methods

• For $2 \times 2$ matrices, the inverse can be calculated using the formula: $A^{-1} = \frac{1}{\det(A)} * \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$, where $A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}$
  • For example, if $A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$, then $A^{-1} = \frac{1}{-2} * \begin{bmatrix} 4 & -2 \\ -3 & 1 \end{bmatrix} = \begin{bmatrix} -2 & 1 \\ \frac{3}{2} & -\frac{1}{2} \end{bmatrix}$
• Singular Value Decomposition (SVD) method: Decompose the matrix $A$ into the product of three matrices, $A = U * \Sigma * V^T$, and then compute $A^{-1} = V * \Sigma^{-1} * U^T$, where $\Sigma^{-1}$ is obtained by reciprocating the non-zero singular values
  • This method is numerically stable and can be used for both square and rectangular matrices

Solving Linear Systems with Matrix Inversion

Solving Systems of Linear Equations

• A system of linear equations $Ax = b$, where $A$ is an invertible matrix, can be solved by multiplying both sides by $A^{-1}$, resulting in $x = A^{-1} * b$
  • For example, consider the system of equations: $\begin{cases} x + 2y = 5 \\ 3x + 4y = 11 \end{cases}$ This can be written in matrix form as $\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 5 \\ 11 \end{bmatrix}$ Multiplying both sides by $A^{-1} = \begin{bmatrix} -2 & 1 \\ \frac{3}{2} & -\frac{1}{2} \end{bmatrix}$, we get: $\begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} -2 & 1 \\ \frac{3}{2} & -\frac{1}{2} \end{bmatrix} \begin{bmatrix} 5 \\ 11 \end{bmatrix} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}$ So, the solution is $x = 1$ and $y = 2$
• The solution obtained using matrix inversion is unique because the inverse of an invertible matrix is unique

Least-Squares Solution and Computational Considerations

• Matrix inversion can be used to find the least-squares solution to an overdetermined system of linear equations
  • For example, consider the system $Ax = b$ where $A$ is an $m \times n$ matrix with $m > n$. The least-squares solution minimizes the Euclidean norm of the residual vector $r = b - Ax$ and is given by $x = (A^T A)^{-1} A^T b$
• In practice, solving systems of linear equations using matrix inversion can be computationally expensive for large matrices, and other methods like LU decomposition or iterative methods may be preferred
  • For example, Gaussian elimination with partial pivoting (LU decomposition) has a time complexity of $O(n^3)$ for an $n \times n$ matrix, while matrix inversion using the same method has a complexity of $O(n^3)$ plus the cost of matrix multiplication, which is also $O(n^3)$ in the naive implementation