2.4 Statistical Interpretation of Entropy

Entropy and Microstates

Statistical Interpretation of Entropy

The statistical interpretation of entropy connects a macroscopic thermodynamic quantity to what's happening at the particle level. Instead of treating entropy as an abstract state function, you can understand it as a direct reflection of how many ways the particles in a system can be arranged.

A microstate is one specific configuration of all the particles in a system, specifying each particle's position, energy level, or quantum state. A macrostate describes the system through macroscopic properties like temperature, pressure, and volume. Many different microstates can correspond to the same macrostate, and the key insight is this: macrostates with more corresponding microstates have higher entropy.

"Disorder" is a common shorthand for entropy, but it's more precise to think of entropy as a measure of multiplicity, the number of microstates consistent with a given macrostate.

Entropy and Natural Processes

Systems tend to evolve toward macrostates that have the largest number of corresponding microstates, simply because those macrostates are overwhelmingly more probable. This is the statistical basis of the second law of thermodynamics. There's no mysterious "force" driving entropy up; it's pure probability.

Consider some familiar examples:

• Mixing of two gases. When a partition between nitrogen and oxygen is removed, the number of spatial configurations available to the combined system is astronomically larger than when the gases are separated. The mixed state dominates statistically.
• Diffusion of ink in water. A drop of ink concentrated in one spot corresponds to very few microstates. Once the ink molecules spread throughout the water, the number of accessible microstates explodes.
• Thermal equilibration. A hot object in contact with a cool room can distribute its energy across far more microstates when energy flows to the surroundings than when it stays concentrated in the hot object.

In every case, the system moves toward the macrostate with the greatest multiplicity, not because it "wants" to, but because that outcome is statistically dominant.

Boltzmann Entropy Formula

Derivation of the Boltzmann Entropy Formula

The Boltzmann entropy formula is:

$S = k_B \ln W$

where $S$ is the entropy, $W$ is the number of microstates accessible to the system, and $k_B = 1.381 \times 10^{-23}$ J/K is the Boltzmann constant.

The logarithmic form isn't arbitrary. It's required by two physical constraints:

1. Entropy must be extensive. If you combine two independent subsystems with $W_1$ and $W_2$ microstates, the total number of microstates is $W_{\text{total}} = W_1 \cdot W_2$. Taking the logarithm converts this product into a sum: $S_{\text{total}} = k_B \ln(W_1 W_2) = k_B \ln W_1 + k_B \ln W_2 = S_1 + S_2$. This is exactly the additive behavior you need for an extensive property.
2. Entropy must increase monotonically with multiplicity. More microstates should always mean more entropy, and $\ln W$ is a monotonically increasing function of $W$.

The Boltzmann constant $k_B$ serves as the bridge between the microscopic counting of states and the macroscopic SI units of entropy (J/K). It ensures that the statistical definition of entropy matches the classical thermodynamic definition (e.g., $\Delta S = q_{\text{rev}}/T$).

Implications of the Boltzmann Entropy Formula

Several important consequences follow from $S = k_B \ln W$:

• Second law, restated. Systems evolve toward macrostates with larger $W$ because those states are more probable. The second law is a statistical certainty for large $N$, not an absolute prohibition against entropy decrease, but fluctuations become negligibly small for macroscopic systems (on the order of $10^{23}$ particles).
• Scaling with system size. Adding more particles increases the number of accessible microstates combinatorially. For $N$ particles, $W$ typically grows exponentially with $N$, so $S \propto N$, confirming extensivity.
• Absolute zero. A perfect crystal at 0 K has exactly one accessible microstate ($W = 1$), giving $S = k_B \ln 1 = 0$. This is the statistical foundation of the third law of thermodynamics.
• Microscopic-macroscopic bridge. The formula lets you compute a measurable thermodynamic quantity (entropy) directly from a count of quantum states, connecting statistical mechanics to classical thermodynamics.

Calculating Entropy

Entropy of Systems with Distinguishable Particles

To apply $S = k_B \ln W$, you first need to count the microstates $W$ for the macrostate of interest.

For a system of $N$ distinguishable particles, each of which can occupy one of two states (think of a lattice of localized spins, each pointing up or down), the total number of microstates is:

$W = 2^N$

The entropy is then:

$S = k_B \ln(2^N) = N k_B \ln 2$

Example: Take 4 distinguishable particles (A, B, C, D), each in state 0 or 1. The total number of microstates is $W = 2^4 = 16$, covering every combination from (0,0,0,0) to (1,1,1,1). The entropy of this system is:

$S = 4 k_B \ln 2 \approx 4(1.381 \times 10^{-23})(0.693) \approx 3.83 \times 10^{-23} \text{ J/K}$

Notice that the entropy scales linearly with $N$, as expected for an extensive quantity.

Entropy of Systems with Indistinguishable Particles

When particles are indistinguishable, swapping two particles doesn't create a new microstate. The counting changes. For $N$ sites with $n$ indistinguishable particles distributed among them (each site either occupied or empty), the number of microstates is the binomial coefficient:

$W = \binom{N}{n} = \frac{N!}{n!(N-n)!}$

The entropy is:

$S = k_B \ln \binom{N}{n}$

Example: A lattice of 6 sites with 3 indistinguishable particles gives:

$W = \binom{6}{3} = \frac{6!}{3! \cdot 3!} = \frac{720}{6 \cdot 6} = 20$

$S = k_B \ln 20 \approx k_B (3.00) \approx 4.14 \times 10^{-23} \text{ J/K}$

Compare this to the distinguishable case: if those 3 particles were distinguishable on 6 sites, you'd count $\frac{6!}{(6-3)!} = 120$ microstates, giving a higher entropy. The distinction between distinguishable and indistinguishable particles matters significantly for getting the correct entropy.

When working through entropy calculations, always:

1. Identify whether particles are distinguishable or indistinguishable.
2. Determine the correct expression for $W$ based on the system's constraints.
3. Apply $S = k_B \ln W$ and simplify using logarithm properties (or Stirling's approximation for large systems).