2.3 Inner and outer products

Inner and outer products are essential tools in vector algebra, offering unique insights into geometric relationships. They help us understand how vectors interact, measure alignment, and represent areas and volumes in multidimensional spaces.

These concepts are fundamental to many applications in physics, computer graphics, and engineering. Inner products reveal vector similarities and projections, while outer products capture oriented areas and volumes, providing a powerful framework for solving complex geometric problems.

Inner products: Geometric interpretation

Vector alignment and similarity

• The inner product of two vectors is a scalar value that measures their alignment or similarity
• Positive inner product indicates vectors pointing in similar directions (acute angle between them)
• Zero inner product means vectors are orthogonal (perpendicular to each other)
• Negative inner product suggests vectors pointing in opposite directions (obtuse angle between them)
• Example: In a 2D space, vectors $a = (1, 1)$ and $b = (1, -1)$ have an inner product of zero, indicating they are orthogonal

Angle and projection

• Geometrically, the inner product of two unit vectors equals the cosine of the angle between them
  • $a \cdot b = |a| |b| \cos(\theta)$, where $\theta$ is the angle between vectors $a$ and $b$
• The inner product's absolute value equals the length of the projection of one vector onto the other, multiplied by the length of the other vector
  • $|a \cdot b| = |a| |proj_b(a)|$, where $proj_b(a)$ is the projection of $a$ onto $b$
• Example: Given vectors $a = (3, 4)$ and $b = (1, 0)$, the projection of $a$ onto $b$ is $(3, 0)$, and $|a \cdot b| = 3$

Cauchy-Schwarz inequality

• The Cauchy-Schwarz inequality states that the absolute value of the inner product of two vectors is less than or equal to the product of their magnitudes
  • $|a \cdot b| \leq |a| |b|$
• Equality holds if and only if one vector is a scalar multiple of the other
• Example: For vectors $a = (1, 2)$ and $b = (3, 4)$, $|a \cdot b| = 11$, which is less than $|a| |b| = \sqrt{5} \sqrt{25} = 5\sqrt{5}$

Inner products: Computation in vector spaces

Euclidean space

• In Euclidean space, the inner product (dot product) of two vectors $a = (a_1, a_2, ..., a_n)$ and $b = (b_1, b_2, ..., b_n)$ is defined as $a \cdot b = a_1b_1 + a_2b_2 + ... + a_nb_n$
• Example: In 3D space, for vectors $a = (1, 2, 3)$ and $b = (4, 5, 6)$, $a \cdot b = 1 \times 4 + 2 \times 5 + 3 \times 6 = 32$

Function spaces

• In function spaces, such as the space of continuous functions on an interval $[a, b]$, the inner product of two functions $f$ and $g$ is defined as the integral of their product over the interval: $\langle f, g \rangle = \int_a^b f(x)g(x) dx$
• Example: For functions $f(x) = x$ and $g(x) = x^2$ on the interval $[0, 1]$, $\langle f, g \rangle = \int_0^1 x \cdot x^2 dx = \frac{1}{4}$

Matrix spaces

• In matrix spaces, the inner product (Frobenius inner product) of two matrices $A$ and $B$ is the sum of the products of their corresponding entries: $\langle A, B \rangle = \sum_{i,j} A_{ij}B_{ij}$
• Example: For matrices $A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$ and $B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}$, $\langle A, B \rangle = 1 \times 5 + 2 \times 6 + 3 \times 7 + 4 \times 8 = 70$

Complex vector spaces

• The inner product of two complex vectors $z = (z_1, z_2, ..., z_n)$ and $w = (w_1, w_2, ..., w_n)$ is defined as $\langle z, w \rangle = z_1^*w_1 + z_2^*w_2 + ... + z_n^*w_n$, where $*$ denotes the complex conjugate
• Example: For complex vectors $z = (1 + i, 2 - i)$ and $w = (3 + 2i, 1 - 3i)$, $\langle z, w \rangle = (1 - i)(3 + 2i) + (2 + i)(1 - 3i) = 8 + 5i$

Outer products: Definition and properties

Bivectors and oriented area

• The outer product of two vectors $a$ and $b$, denoted as $a \wedge b$, is a bivector that represents the oriented area spanned by the two vectors
• The magnitude of the outer product equals the area of the parallelogram formed by the two vectors
  • $|a \wedge b| = |a| |b| \sin(\theta)$, where $\theta$ is the angle between vectors $a$ and $b$
• Example: For vectors $a = (1, 0)$ and $b = (0, 1)$, $a \wedge b$ represents a unit square in the $xy$-plane with a counterclockwise orientation

Anticommutativity and distributivity

• The outer product is anticommutative: $a \wedge b = -b \wedge a$, meaning that swapping the order of the vectors changes the sign of the result
• The outer product is distributive over addition: $(a + b) \wedge c = a \wedge c + b \wedge c$ and $a \wedge (b + c) = a \wedge b + a \wedge c$
• Example: For vectors $a = (1, 0)$, $b = (0, 1)$, and $c = (1, 1)$, $(a + b) \wedge c = a \wedge c + b \wedge c = (1, -1) + (1, 1) = (2, 0)$

Self-product and associativity

• The outer product of a vector with itself is zero: $a \wedge a = 0$
• The outer product is associative: $(a \wedge b) \wedge c = a \wedge (b \wedge c)$
• Example: For vectors $a = (1, 0, 0)$, $b = (0, 1, 0)$, and $c = (0, 0, 1)$, $(a \wedge b) \wedge c = a \wedge (b \wedge c) = (0, 0, 1)$, representing a unit volume in the $xyz$-space with a right-handed orientation

Inner vs Outer products: Applications in geometry

Angles, projections, and collision detection

• Use the inner product to determine the angle between two vectors, which can be useful in collision detection, computer graphics, and physics simulations
  • $\cos(\theta) = \frac{a \cdot b}{|a| |b|}$
• Apply the inner product to find the projection of one vector onto another, which is essential in solving problems involving forces, shadows, and least-squares approximations
  • $proj_b(a) = \frac{a \cdot b}{|b|^2} b$
• Example: In a game engine, the inner product can be used to check if two objects are colliding by comparing the angle between their surface normal vectors

Areas, normals, and cross products

• Employ the outer product to calculate the area of a parallelogram or triangle spanned by two vectors in 2D or 3D space
  • Parallelogram area: $A = |a \wedge b|$
  • Triangle area: $A = \frac{1}{2} |a \wedge b|$
• Use the outer product to determine the normal vector to a plane or hyperplane, which is crucial in computer graphics, physics, and engineering applications
  • For vectors $a$ and $b$ spanning a plane, the normal vector is given by $n = a \wedge b$
• In 3D space, the outer product is related to the cross product: $a \times b = -I (a \wedge b)$, where $I$ is the pseudoscalar
• Example: In computer graphics, the outer product can be used to calculate surface normals for shading and lighting calculations

Complex geometric problems

• Combine inner and outer products to solve more complex geometric problems, such as finding the distance between a point and a line or plane, or determining the intersection of lines, planes, or higher-dimensional objects
• Example: To find the distance between a point $p$ and a plane defined by a point $q$ and a normal vector $n$, use the formula $d = \frac{(p - q) \cdot n}{|n|}$
• Example: To determine the intersection point of two lines in 3D space, defined by points $p_1$, $p_2$ and $q_1$, $q_2$, respectively, solve the equation $(p_1 + t(p_2 - p_1)) \wedge (q_1 + s(q_2 - q_1)) = 0$ for $t$ and $s$