5.4 Statistical interpretation of entropy

The statistical interpretation of entropy connects microscopic particle arrangements to macroscopic thermodynamic properties. Instead of treating entropy as an abstract quantity, this framework explains why systems evolve toward equilibrium by counting the number of ways particles can be arranged. It's the bridge between molecular behavior and the second law.

Entropy and Microstates

A microstate is one specific arrangement of all the particles in a system, including their positions and momenta. A macrostate is the set of macroscopic properties you actually measure (temperature, pressure, volume). Many different microstates can correspond to the same macrostate.

Entropy quantifies how many microstates are consistent with a given macrostate. The more microstates available, the higher the entropy. This relationship is logarithmic, not linear, which matters once you start working with real numbers. For a system of even modest size (say, a mole of gas), the number of microstates is astronomically large, often on the order of $10^{10^{23}}$. The logarithm brings these numbers into a manageable range.

Why This Relationship Matters

The entropy-microstate connection gives a microscopic explanation for macroscopic behavior:

• It explains why systems tend toward equilibrium: equilibrium corresponds to the macrostate with the overwhelmingly largest number of microstates.
• It explains irreversibility: a system doesn't return to a low-entropy state because the probability of spontaneously finding all particles in a highly ordered arrangement is vanishingly small.
• It provides the foundation for calculating entropy from first principles using the Boltzmann equation.

Calculating Entropy with the Boltzmann Equation

The Equation and Its Components

The Boltzmann equation relates entropy directly to the number of microstates:

$S = k_B \ln W$

where:

• $S$ is the entropy of the system
• $k_B$ is the Boltzmann constant, $1.380649 \times 10^{-23} \text{ J/K}$
• $W$ is the number of microstates accessible to the system

The Boltzmann constant acts as a unit-conversion factor. It ensures entropy comes out in J/K, matching the macroscopic thermodynamic definition. The natural logarithm captures the fact that entropy is an extensive property: if you combine two independent systems, $W_{\text{total}} = W_1 \times W_2$, and $\ln(W_1 \times W_2) = \ln W_1 + \ln W_2$, so the entropies add, exactly as they should.

Applying the Equation

To use the Boltzmann equation, follow these steps:

1. Define the system and identify the relevant degrees of freedom (positions, energy levels, spin states, etc.).
2. Count the microstates $W$. For simple systems, this is a combinatorial calculation. For example, distributing $N$ indistinguishable particles among $g$ energy levels uses the appropriate multiplicity formula.
3. Substitute into the equation. Plug $W$ and $k_B$ into $S = k_B \ln W$.

Example: Consider a two-state spin system with $N = 4$ particles, where each particle can be spin-up or spin-down. The total number of microstates is $W = 2^4 = 16$. The entropy is:

$S = k_B \ln 16 = k_B \times 2.773 = 3.83 \times 10^{-23} \text{ J/K}$

The macrostate where 2 spins are up and 2 are down has the most microstates ($W = 6$), so it has the highest entropy among the individual macrostates. This is a simple illustration of why systems favor the most "mixed" configurations.

Disorder and the Second Law

What "Disorder" Actually Means Here

The word "disorder" is common shorthand, but it can be misleading. More precisely, higher entropy means a larger number of accessible microstates. A gas expanded to fill a container isn't "messy" in an everyday sense; it simply has far more spatial arrangements available than when it was compressed into one corner.

• A system with high entropy has many accessible microstates.
• A system with low entropy is restricted to relatively few microstates.

Connection to the Second Law

The second law states that the total entropy of an isolated system never decreases. The statistical interpretation makes this almost obvious:

• The macrostate with the most microstates is overwhelmingly the most probable.
• As a system evolves, it samples microstates randomly. Since the vast majority of microstates belong to the high-entropy macrostate, the system will almost certainly be found there.
• A spontaneous decrease in entropy would require the system to find its way into a tiny subset of microstates. For macroscopic systems, the probability of this is effectively zero (though not exactly zero, which is a subtle but real distinction).

This is why gases mix spontaneously, heat flows from hot to cold, and you never see a broken egg reassemble. The reverse processes aren't forbidden by any fundamental law of motion; they're just overwhelmingly improbable.

Entropy as a Probability Distribution

The Gibbs Entropy Formula

The Boltzmann equation assumes all accessible microstates are equally probable, which holds for an isolated system at equilibrium. For more general situations (non-uniform distributions, systems in contact with a heat bath), the Gibbs entropy formula applies:

$S = -k_B \sum_i p_i \ln p_i$

where $p_i$ is the probability of the system being in microstate $i$. When all $W$ microstates are equally likely, $p_i = 1/W$ for each, and the Gibbs formula reduces to:

$S = -k_B \sum_{i=1}^{W} \frac{1}{W} \ln \frac{1}{W} = -k_B \cdot W \cdot \frac{1}{W} \ln \frac{1}{W} = k_B \ln W$

This confirms that the Boltzmann equation is a special case of the Gibbs formula.

Factors That Shape the Distribution

Several variables determine how probability is spread across microstates:

• Temperature: At higher temperatures, the system has more thermal energy, so more microstates become energetically accessible. The probability distribution flattens out, and entropy increases.
• Volume: A larger volume provides more positional microstates for gas-phase particles, increasing $W$ and therefore $S$.
• Number of particles: More particles means exponentially more ways to distribute energy and positions, dramatically increasing the number of microstates.

As a system approaches equilibrium, the probability distribution evolves toward the one that maximizes entropy. This is entirely consistent with the second law: the equilibrium distribution is the one spread across the most microstates, making it the most probable outcome by an enormous margin.