6.4 The statistical interpretation of entropy

Statistical Interpretation of Entropy

Entropy isn't just about heat and energy. It's a measure of disorder at the microscopic level, counting the number of ways particles can arrange themselves. The more ways they can be arranged, the higher the entropy.

This statistical view connects directly to the thermodynamic definition of entropy. It explains why heat flows from hot to cold, why certain processes are irreversible, and how systems evolve over time.

Microscopic Interpretation of Entropy

At the microscopic level, entropy measures how many different configurations the particles in a system can take on. Each specific arrangement of particles (their positions and momenta) is called a microstate. A system with more accessible microstates has higher entropy because there are more ways to distribute energy among its particles.

Boltzmann captured this idea in one equation:

$S = k_B \ln \Omega$

• $S$ is the entropy of the system
• $k_B$ is the Boltzmann constant ($1.38 \times 10^{-23}$ J/K)
• $\Omega$ is the number of microstates available to the system

The natural logarithm matters here. Because $\Omega$ can be astronomically large (think $10^{23}$ particles each with many possible states), the logarithm keeps $S$ at a manageable scale. It also makes entropy additive: if you combine two independent systems with $\Omega_1$ and $\Omega_2$ microstates, the total number of microstates is $\Omega_1 \times \Omega_2$, and the total entropy is $S_1 + S_2$ because $\ln(\Omega_1 \Omega_2) = \ln \Omega_1 + \ln \Omega_2$.

Statistical vs. Thermodynamic Entropy

The thermodynamic definition of entropy is based on heat transfer in a reversible process:

$dS = \frac{dQ_{\text{rev}}}{T}$

• $dS$ is the infinitesimal change in entropy
• $dQ_{\text{rev}}$ is the heat transferred reversibly
• $T$ is the absolute temperature

This definition came first historically, but the statistical definition ($S = k_B \ln \Omega$) gives it a physical meaning. The two are fully consistent with each other. Here's how to see the connection:

• When you transfer heat $dQ$ to a system at temperature $T$, you increase the energy available to the particles. That opens up new energy levels and arrangements, increasing $\Omega$.
• The thermodynamic formula tells you how much entropy changes during a process. The statistical formula tells you why it changes: because the number of accessible microstates grew.

Notice that dividing by $T$ in the thermodynamic formula also makes physical sense statistically. A system already at high temperature has many accessible microstates, so adding a small amount of heat doesn't change $\Omega$ by much. A cold system has fewer accessible microstates, so the same heat addition creates a proportionally larger increase in $\Omega$.

Entropy Calculations for Simple Configurations

To calculate entropy statistically, you follow a clear process:

1. Define the system. Identify the particles, their energy levels, and any constraints (total energy, volume, particle number).
2. Count the microstates ($\Omega$). Use combinatorial methods to determine how many distinct ways the particles can be arranged among the available states, consistent with the constraints.
3. Apply the Boltzmann equation. Plug $\Omega$ into $S = k_B \ln \Omega$.

For example, consider distributing $q$ identical energy quanta among $N$ oscillators in an Einstein solid. The number of microstates is:

$\Omega = \binom{q + N - 1}{q} = \frac{(q + N - 1)!}{q!(N - 1)!}$

If you have $N = 3$ oscillators and $q = 4$ quanta, then $\Omega = \frac{6!}{4! \cdot 2!} = 15$, and the entropy is $S = k_B \ln 15 \approx 2.71 \, k_B$.

For an ideal gas, the counting is more involved because you need to account for the number of ways particles can be distributed across position and momentum states within a given volume and energy. The result connects to the Sackur-Tetrode equation, but the underlying logic is the same: count microstates, then take the logarithm.

Entropy and System Microstates

A macrostate describes the system's bulk properties (temperature, pressure, volume). Many different microstates can correspond to the same macrostate. The macrostate with the most microstates is the most probable one.

This is the statistical basis of the Second Law of Thermodynamics: an isolated system naturally evolves toward the macrostate with the largest number of microstates, which is the state of maximum entropy. Entropy doesn't increase because of some mysterious force. It increases because the high-entropy macrostate is overwhelmingly more probable.

Consider a gas confined to one half of a box. When you remove the partition:

• The macrostate "all molecules in the left half" has relatively few microstates (particles are restricted to half the volume).
• The macrostate "molecules spread throughout the box" has vastly more microstates.
• The gas expands not because it "wants" to, but because random molecular motion makes the spread-out configuration almost certain.

For any macroscopic system (say $10^{23}$ particles), the number of microstates for the equilibrium macrostate is so astronomically larger than for any non-equilibrium macrostate that spontaneous decreases in entropy are effectively impossible. They're not forbidden by any law of physics at the microscopic level, but the probability is so vanishingly small that you'd have to wait far longer than the age of the universe to see one happen.

This is why the Second Law holds: $\Delta S \geq 0$ for an isolated system, with equality only for reversible processes.