Fundamental Tensor Operations

Why This Matters

Tensor operations aren't just abstract mathematical machinery—they're the language that lets you express physical laws in a form that works in any coordinate system. When you're studying general relativity, continuum mechanics, or even modern machine learning, you're being tested on whether you understand how tensors combine, contract, and transform. The operations covered here—addition, multiplication, contraction, differentiation, and decomposition—show up repeatedly in derivations and problem sets because they're the fundamental building blocks for everything else.

Don't just memorize what each operation does; know why you'd choose one over another. Can you explain when contraction is more useful than an outer product? Do you understand why transformation rules matter for physical consistency? These conceptual connections are what separate students who can solve problems from those who just recognize formulas. Master the underlying logic, and the calculations become straightforward.

---

Building Tensors: Combination Operations

These operations let you construct new tensors from existing ones. The key principle is that combining tensors of compatible types produces predictable results—the output type depends entirely on the input types and how you combine them.

Tensor Addition and Subtraction

• Same type and order required—you can only add or subtract tensors that share identical index structures (both must be, say, type $(1,2)$ tensors)
• Element-wise operation preserves tensor order; the result has the same rank as the inputs
• Commutative and associative properties hold, making tensor addition behave like familiar vector addition

Tensor Multiplication (Outer Product)

• Order increases additively—combining a rank-$m$ tensor with a rank-$n$ tensor yields a rank-$(m+n)$ tensor
• Not commutative; $A \otimes B \neq B \otimes A$ because index ordering matters for the resulting tensor structure
• Builds higher-order objects from simpler pieces, essential for constructing tensor products of vector spaces

---

Reducing Complexity: Contraction Operations

Contraction is how you extract meaningful quantities from tensors by summing over paired indices. This is the generalization of the dot product to arbitrary tensor ranks—it's how physics "collapses" tensor information into scalars or lower-rank objects.

Tensor Contraction

• Reduces tensor order by 2 for each pair of contracted indices (one upper, one lower)
• Summing over repeated indices implements the Einstein summation convention: $T^i_{ij} = \sum_i T^i_{ij}$
• Extracts invariants like the trace; contraction is how you get coordinate-independent scalar quantities from tensors

Tensor Inner Product

• Produces a scalar by contracting all indices between two tensors of the same order
• Generalizes the dot product; for vectors, this recovers $\vec{A} \cdot \vec{B} = A^i B_i$
• Requires index compatibility—you're essentially performing full contraction between the two tensor arguments

---

Structural Rearrangements: Symmetry Operations

These operations reorganize tensor components to reveal or impose symmetry properties. Understanding symmetry is critical because many physical tensors have built-in symmetry constraints that simplify calculations dramatically.

Tensor Transpose

• Swaps index positions—for a second-order tensor, $T^{\top}_{ij} = T_{ji}$, exchanging rows and columns
• Defines symmetry classification; a tensor equals its transpose if symmetric, equals negative transpose if antisymmetric
• Generalizes to higher orders by specifying which pair of indices to exchange

Tensor Symmetrization and Antisymmetrization

• Symmetrization averages over all index permutations: $T_{(ij)} = \frac{1}{2}(T_{ij} + T_{ji})$
• Antisymmetrization creates sign-alternating combinations: $T_{[ij]} = \frac{1}{2}(T_{ij} - T_{ji})$
• Physical significance—stress tensors are symmetric, electromagnetic field tensors are antisymmetric; these properties encode conservation laws

---

Calculus on Tensor Fields: Differentiation and Integration

When tensors vary across space or time, you need tensor calculus to describe rates of change and accumulated quantities. The challenge is ensuring derivatives and integrals remain tensors themselves.

Tensor Differentiation

• Covariant derivative $\nabla_\mu T^\nu$ accounts for how basis vectors change, adding connection terms beyond ordinary partial derivatives
• Essential for curved spaces—in general relativity, ordinary derivatives don't transform as tensors; covariant derivatives do
• Produces tensors of higher order; differentiating a rank-$n$ tensor yields a rank-$(n+1)$ tensor field

Tensor Integration

• Integrates over domains using appropriate volume elements that transform correctly under coordinate changes
• Requires metric information—the volume element $\sqrt{|g|} \, d^n x$ ensures coordinate independence
• Applications in physics include computing total charge, momentum flux, and energy in field theories

---

Coordinate Independence: Transformation Rules

The defining property of tensors is how they transform under coordinate changes. This isn't just a technicality—it's what makes tensor equations valid physics in any reference frame.

Tensor Transformation Under Coordinate Changes

• Transformation law involves the Jacobian: $T'^{\mu\nu} = \frac{\partial x'^\mu}{\partial x^\alpha} \frac{\partial x'^\nu}{\partial x^\beta} T^{\alpha\beta}$ for contravariant indices
• Covariant indices transform with the inverse Jacobian, ensuring the distinction between upper and lower indices
• Physical consistency—equations written in tensor form automatically hold in all coordinate systems, which is why tensors are the language of relativity

---

Simplification and Analysis: Decomposition Methods

Tensor decomposition breaks complex tensors into simpler pieces, revealing structure and enabling efficient computation. This is where pure mathematics meets practical data science.

Tensor Decomposition

• CP decomposition (CANDECOMP/PARAFAC) expresses a tensor as a sum of rank-one tensors: $T = \sum_r \lambda_r \, a_r \otimes b_r \otimes c_r$
• Tucker decomposition uses a core tensor multiplied by factor matrices along each mode, offering more flexibility than CP
• Applications span data compression, latent factor discovery, and dimensionality reduction in machine learning

---

Quick Reference Table

---

Self-Check Questions

1. Which two operations both reduce tensor rank, and what distinguishes their outputs?

2. If you need to construct a rank-4 tensor from two vectors and a matrix, which operation(s) would you use, and what would the resulting index structure be?

3. Compare and contrast symmetrization and antisymmetrization: how does each operation behave under index exchange, and give one physical example where each is relevant.

4. Why does tensor differentiation in curved spaces require the covariant derivative rather than ordinary partial derivatives? What goes wrong if you use partial derivatives?

5. An FRQ asks you to show that a physical law is coordinate-independent. Which tensor operation's transformation rules would you invoke, and what mathematical object appears in those rules?