1.6 Statistical ensembles

Concept of statistical ensembles

A statistical ensemble is the central tool that lets you go from "what are all the particles doing?" to "what does the thermometer read?" Instead of tracking every particle individually, you consider a huge collection of hypothetical copies of your system, each in a different possible microstate, and use probability to extract macroscopic predictions.

Definition and purpose

Think of an ensemble as a mental collection of copies of your system, one for every possible microstate the system could occupy. You never build these copies in a lab. They exist as a theoretical device that lets you replace impossible particle-by-particle tracking with a probability-weighted average.

• Each copy in the ensemble represents one allowed microstate of the system
• Macroscopic properties (temperature, pressure, etc.) come from averaging over all these copies
• This probabilistic approach is what makes statistical mechanics tractable for systems with $\sim 10^{23}$ particles

Types of ensembles

Each ensemble type corresponds to different physical constraints on the system. The choice depends on what your system can exchange with its surroundings.

• Microcanonical ensemble: Isolated system. Energy $E$, volume $V$, and particle number $N$ are all fixed. Nothing goes in or out.
• Canonical ensemble: System in contact with a heat bath. Temperature $T$, $V$, and $N$ are fixed, but energy can fluctuate.
• Grand canonical ensemble: System exchanges both energy and particles with a reservoir. $T$, $V$, and chemical potential $\mu$ are fixed.
• Isothermal-isobaric ensemble: System at constant temperature and pressure. $T$, $P$, and $N$ are fixed; volume can fluctuate.

Ensemble averages

An ensemble average is how you actually extract a measurable quantity from the ensemble. You sum a property over all microstates, weighting each by its probability:

$\langle A \rangle = \sum_i p_i A_i$

where $p_i$ is the probability of microstate $i$ and $A_i$ is the value of the observable in that microstate. This yields expectation values for thermodynamic variables like internal energy, pressure, and magnetization. The key point: the ensemble average is the theoretical prediction for what you'd measure in the lab.

Microcanonical ensemble

The microcanonical ensemble describes a completely isolated system with fixed $E$, $V$, and $N$. It's the most fundamental ensemble and the logical starting point because its core assumption (equal probability of microstates) underpins all of statistical mechanics.

Isolated systems

An isolated system has no exchange of energy or matter with its surroundings. Total energy is strictly conserved, and both volume and particle number are fixed. This is an idealization, but it's useful for deriving foundational results and for studying how closed systems approach equilibrium.

Equal a priori probability

This is the fundamental postulate of statistical mechanics: all accessible microstates of an isolated system at a given energy are equally probable. There's no derivation of this from deeper principles; it's an axiom justified by its success.

• "Accessible" means consistent with the macroscopic constraints ($E$, $V$, $N$)
• Because every microstate is equally likely, probability reduces to counting: $p_i = 1/\Omega$
• This postulate leads directly to the principle of maximum entropy

Entropy and multiplicity

With equal a priori probability established, entropy follows naturally:

$S = k_B \ln \Omega$

Here $\Omega$ is the multiplicity, the total number of microstates accessible at energy $E$. Boltzmann's constant $k_B = 1.38 \times 10^{-23} \text{ J/K}$ sets the scale, connecting the dimensionless microstate count to thermodynamic units.

The second law of thermodynamics emerges from this: systems evolve toward the macrostate with the largest $\Omega$, because that's overwhelmingly the most probable configuration.

Canonical ensemble

The canonical ensemble is probably the one you'll use most. It describes a system that can exchange energy with a large heat bath, keeping temperature constant while energy fluctuates. Most real experiments happen at controlled temperature, so this ensemble maps directly onto common lab conditions.

Systems in thermal equilibrium

When your system is in thermal contact with a heat bath (a reservoir so large its temperature doesn't change), energy flows back and forth until equilibrium is reached. The probability of finding the system in microstate $i$ with energy $E_i$ follows the Boltzmann distribution:

$p_i = \frac{1}{Z} e^{-\beta E_i}$

where $\beta = 1/(k_B T)$. Higher-energy states are exponentially suppressed relative to lower-energy ones, with the suppression controlled by temperature.

Partition function

The partition function $Z$ is the normalization constant that makes probabilities sum to one, but it turns out to encode all thermodynamic information about the system:

$Z = \sum_i e^{-\beta E_i}$

The sum runs over every microstate $i$. Once you have $Z$, you can derive essentially any equilibrium property. For a continuous system, the sum becomes an integral over phase space.

Helmholtz free energy

The natural thermodynamic potential for the canonical ensemble is the Helmholtz free energy:

$F = -k_B T \ln Z$

This single equation connects the partition function to thermodynamics. At fixed $T$ and $V$, equilibrium corresponds to the minimum of $F$. From $F$ you can get entropy ($S = -\partial F/\partial T$), pressure ($P = -\partial F/\partial V$), and other quantities through standard thermodynamic derivatives.

Grand canonical ensemble

The grand canonical ensemble handles open systems that exchange both energy and particles with a reservoir. Temperature, volume, and chemical potential are held fixed, while both energy and particle number fluctuate. This is the natural framework for systems where particles can enter or leave, like gas molecules adsorbing onto a surface or electrons in a metal.

Open systems

• Both energy and particles flow between the system and reservoir
• Volume remains fixed
• The particle number $N$ fluctuates around an average value $\langle N \rangle$
• Physical examples: gas adsorption on surfaces, electron gases in metals, chemical reactions in solution

Chemical potential

The chemical potential $\mu$ governs particle exchange the same way temperature governs energy exchange. It's defined as:

$\mu = \left(\frac{\partial F}{\partial N}\right)_{T,V}$

Particles flow from regions of high $\mu$ to low $\mu$ until equilibrium is reached (just as heat flows from high $T$ to low $T$). In the grand canonical ensemble, $\mu$ is fixed by the reservoir.

Grand partition function

The grand partition function sums over all possible particle numbers and all microstates at each particle number:

$\Xi = \sum_{N=0}^{\infty} e^{\beta \mu N} Z_N$

where $Z_N$ is the canonical partition function for $N$ particles. The factor $e^{\beta \mu N}$ weights each particle-number sector by the chemical potential. From $\Xi$ you can compute $\langle N \rangle$, particle number fluctuations, and the grand potential $\Omega = -k_B T \ln \Xi$.

Ensemble equivalence

A natural question arises: if different ensembles fix different variables, do they give different answers? For large systems, the answer is no. This is the principle of ensemble equivalence.

Thermodynamic limit

Ensemble equivalence holds in the thermodynamic limit, where $N \to \infty$ and $V \to \infty$ while the density $N/V$ stays constant. In this limit:

• Relative fluctuations in extensive quantities (energy, particle number) shrink as $1/\sqrt{N}$
• A system of $10^{23}$ particles has relative energy fluctuations of order $10^{-12}$, which is unmeasurably small
• All ensembles converge to the same macroscopic predictions

This is why you're free to pick whichever ensemble makes the math easiest for a given problem.

Fluctuations in ensembles

Each ensemble permits different quantities to fluctuate, and this is what distinguishes them microscopically:

• Microcanonical: Energy is exactly fixed. No fluctuations in $E$, $V$, or $N$.
• Canonical: Energy fluctuates; $N$ and $V$ are fixed. Energy fluctuations satisfy $\langle (\Delta E)^2 \rangle = k_B T^2 C_V$.
• Grand canonical: Both $E$ and $N$ fluctuate. Particle number fluctuations satisfy $\langle (\Delta N)^2 \rangle = k_B T \left(\partial \langle N \rangle / \partial \mu\right)_{T,V}$.

All these fluctuations scale as $1/\sqrt{N}$ relative to the mean, which is why they vanish in the thermodynamic limit.

Ensemble vs. time averages

In the lab, you measure a property by averaging over time. In theory, you average over the ensemble. The ergodic hypothesis states these two averages are equal for systems observed over sufficiently long times.

• A system is ergodic if its trajectory in phase space eventually visits all accessible microstates
• Most equilibrium systems satisfy this condition
• Notable exceptions exist: glasses, systems with broken ergodicity, and metastable states can remain trapped in a subset of phase space

Applications of ensembles

Ideal gas systems

The ideal gas is the classic test case for ensemble methods:

• The canonical partition function for $N$ non-interacting particles in a box yields the equation of state $PV = Nk_BT$
• The microcanonical treatment naturally produces the Maxwell-Boltzmann speed distribution
• The grand canonical ensemble is the right tool for gas adsorption problems, where the number of adsorbed molecules fluctuates

Magnetic systems

The Ising model treats a lattice of spins that can point up or down, with nearest-neighbor interactions. Studied in the canonical ensemble, it exhibits:

• A phase transition from paramagnet to ferromagnet below a critical temperature
• Critical phenomena (diverging susceptibility, power-law correlations) near the transition
• Exact solutions in 1D (no phase transition) and 2D (Onsager's solution)

Quantum statistical mechanics

When quantum effects matter (low temperatures, light particles, high densities), ensemble theory extends naturally:

• The canonical ensemble uses the density matrix $\rho = e^{-\beta H}/Z$
• The grand canonical ensemble for indistinguishable particles yields Fermi-Dirac statistics (fermions) and Bose-Einstein statistics (bosons)
• Applications include electron gases in metals, superconductivity, superfluidity, and Bose-Einstein condensation

Mathematical foundations

Phase space and microstates

For a classical system of $N$ particles in 3D, the phase space has $6N$ dimensions: 3 position coordinates and 3 momentum coordinates per particle. Each point in this space specifies a complete microstate.

• A microstate is one specific configuration of all positions and momenta: $(q_1, \ldots, q_{3N}, p_1, \ldots, p_{3N})$
• For quantum systems, microstates are vectors in Hilbert space, and phase space is replaced by a discrete (or continuous) set of quantum states

Liouville's theorem

Liouville's theorem states that the phase-space distribution function $\rho$ is conserved along trajectories of Hamiltonian dynamics:

$\frac{\partial \rho}{\partial t} + \{\rho, H\} = 0$

where $\{\ ,\ \}$ denotes the Poisson bracket. Physically, this means phase-space volume is incompressible under time evolution. This is crucial because it guarantees that a uniform distribution over an energy surface (the microcanonical ensemble) remains uniform, justifying the use of phase-space averages.

Ergodic hypothesis

The ergodic hypothesis asserts that over long times, a system's trajectory passes through all accessible microstates. This lets you replace a time average (what you measure) with an ensemble average (what you calculate).

• Strictly proving ergodicity is extremely difficult for most systems
• The hypothesis fails for systems with long-lived metastable states, glasses, and integrable systems with too many conserved quantities
• Despite these caveats, it works remarkably well for the vast majority of equilibrium systems

Thermodynamic properties

Derivation from ensembles

Once you have the appropriate partition function, thermodynamic quantities follow systematically:

• Internal energy: $\langle E \rangle = -\frac{\partial \ln Z}{\partial \beta}$
• Entropy: $S = k_B(\ln Z + \beta \langle E \rangle)$ or equivalently $S = -\frac{\partial F}{\partial T}$
• Pressure: $P = -\left(\frac{\partial F}{\partial V}\right)_{T,N} = \frac{1}{\beta}\frac{\partial \ln Z}{\partial V}$
• Heat capacity: $C_V = \frac{\partial \langle E \rangle}{\partial T} = k_B \beta^2 \langle (\Delta E)^2 \rangle$

That last relation is worth noting: heat capacity is directly proportional to energy fluctuations in the canonical ensemble.

Fluctuations and response

A deep result in statistical mechanics is that fluctuations in equilibrium are directly tied to how the system responds to perturbations:

• The fluctuation-dissipation theorem relates equilibrium fluctuations to linear response functions (e.g., susceptibility, compressibility)
• Susceptibilities are second derivatives of thermodynamic potentials (e.g., $\chi = -\partial^2 F / \partial h^2$ for magnetic susceptibility)
• The Einstein relation $D = \mu k_B T$ connects the diffusion coefficient to mobility
• Onsager reciprocal relations follow from microscopic time-reversal symmetry

Thermodynamic potentials

Each ensemble has a natural thermodynamic potential:

Quantum ensembles

At low temperatures or for microscopic systems, quantum effects become essential: energy levels are discrete, particles can be indistinguishable, and the uncertainty principle constrains what states are accessible.

Density matrix formalism

The density matrix (or density operator) generalizes the notion of a quantum state to include statistical mixtures:

$\rho = \sum_i p_i |\psi_i\rangle\langle\psi_i|$

• A pure state has $\rho = |\psi\rangle\langle\psi|$ (one term, $p = 1$)
• A mixed state is a probabilistic combination of pure states
• Expectation values are computed as $\langle A \rangle = \text{Tr}(\rho A)$
• The von Neumann entropy is $S = -k_B \text{Tr}(\rho \ln \rho)$

Quantum canonical ensemble

For a quantum system in thermal equilibrium at temperature $T$:

$\rho = \frac{1}{Z} e^{-\beta H}$

where $H$ is the Hamiltonian operator and $Z = \text{Tr}(e^{-\beta H})$. This is the quantum analog of the Boltzmann distribution. For systems of indistinguishable particles, the requirement of proper symmetrization leads to:

• Fermi-Dirac statistics for fermions (half-integer spin): $\langle n_i \rangle = \frac{1}{e^{\beta(E_i - \mu)} + 1}$
• Bose-Einstein statistics for bosons (integer spin): $\langle n_i \rangle = \frac{1}{e^{\beta(E_i - \mu)} - 1}$

Quantum grand canonical ensemble

The quantum grand canonical ensemble handles open quantum systems where particle number fluctuates. The density matrix becomes:

$\rho = \frac{1}{\Xi} e^{-\beta(H - \mu \hat{N})}$

This framework naturally accommodates particle creation and annihilation, making it the standard starting point for quantum field theory at finite temperature and condensed matter many-body physics.

Ensemble theory in practice

Molecular dynamics simulations

Molecular dynamics (MD) simulates the time evolution of a many-particle system by numerically integrating Newton's equations. The choice of ensemble determines the simulation setup:

1. Microcanonical (NVE): Integrate equations of motion with no external coupling. Energy is conserved by construction.
2. Canonical (NVT): Attach a thermostat (e.g., Nosé-Hoover) that adds or removes kinetic energy to maintain constant temperature.
3. Isothermal-isobaric (NPT): Add both a thermostat and a barostat to control temperature and pressure, allowing volume to fluctuate.

These methods are widely used in materials science, biophysics, and drug design.

Monte Carlo methods

Monte Carlo methods sample the ensemble distribution stochastically rather than following deterministic trajectories:

1. Start from an initial configuration
2. Propose a random move (e.g., displace a particle)
3. Accept or reject the move based on the Metropolis criterion: accept if $\Delta E < 0$; if $\Delta E > 0$, accept with probability $e^{-\beta \Delta E}$
4. Repeat to generate a sequence of configurations sampled from the Boltzmann distribution

Grand canonical Monte Carlo extends this by also proposing particle insertions and deletions, sampling fluctuations in $N$.

Importance sampling

Naive random sampling of phase space is hopelessly inefficient because the vast majority of configurations have negligible Boltzmann weight. Importance sampling solves this by biasing the sampling toward high-probability regions.

• The Metropolis algorithm is itself an importance sampling scheme: it generates configurations with probability proportional to $e^{-\beta E}$
• More advanced techniques (umbrella sampling, Wang-Landau, replica exchange) target rare events and free energy landscapes
• Without importance sampling, simulations of realistic systems would be computationally infeasible