Boltzmann's entropy formula

In statistical mechanics, Boltzmann's equation (also known as the Boltzmann–Planck equation) is a probability equation relating the entropy $S$ , also written as $S_{\mathrm {B} }$ , of an ideal gas to the multiplicity (commonly denoted as $\Omega$ or $W$ ), the number of real microstates corresponding to the gas's macrostate:

S=k_{\mathrm {B} }\ln W

1

where $k_{\mathrm {B} }$ is the Boltzmann constant (also written as simply $k$ ) and equal to 1.380649 × 10⁻²³ J/K, and $\ln$ is the natural logarithm function (or log base e, as in the image above).

In short, the Boltzmann formula shows the relationship between entropy and the number of ways the atoms or molecules of a certain kind of thermodynamic system can be arranged.

History

The equation was originally formulated by Ludwig Boltzmann between 1872 and 1875, but later put into its current form by Max Planck in about 1900.^[2]^[3] To quote Planck, "the logarithmic connection between entropy and probability was first stated by L. Boltzmann in his kinetic theory of gases".^[4]

A 'microstate' is a state specified in terms of the constituent particles of a body of matter or radiation that has been specified as a macrostate in terms of such variables as internal energy and pressure. A macrostate is experimentally observable, with at least a finite extent in spacetime. A microstate can be instantaneous, or can be a trajectory composed of a temporal progression of instantaneous microstates. In experimental practice, such are scarcely observable. The present account concerns instantaneous microstates.

The value of $W$ was originally intended to be proportional to the Wahrscheinlichkeit (the German word for probability) of a macroscopic state for some probability distribution of possible microstates—the collection of (unobservable microscopic single particle) "ways" in which the (observable macroscopic) thermodynamic state of a system can be realized by assigning different positions and momenta to the respective molecules.

There are many instantaneous microstates that apply to a given macrostate. Boltzmann considered collections of such microstates. For a given macrostate, he called the collection of all possible instantaneous microstates of a certain kind by the name monode, for which Gibbs' term ensemble is used nowadays. For single particle instantaneous microstates, Boltzmann called the collection an ergode. Subsequently, Gibbs called it a microcanonical ensemble, and this name is widely used today, perhaps partly because Bohr was more interested in the writings of Gibbs than of Boltzmann.^[5]

Interpreted in this way, Boltzmann's formula is the most basic formula for the thermodynamic entropy. Boltzmann's paradigm was an ideal gas of $N$ identical particles, of which $N i$ are in the $i$ -th microscopic condition (range) of position and momentum. For this case, the probability of each microstate of the system is equal, so it was equivalent for Boltzmann to calculate the number of microstates associated with a macrostate. $W$ was historically misinterpreted as literally meaning the number of microstates, and that is what it usually means today. $W$ can be counted using the formula for permutations

W={\frac {N!}{\prod _{i}N_{i}!}}

2

where $i$ ranges over all possible molecular conditions and " $!$ " denotes factorial. The "correction" in the denominator is due to the fact that identical particles in the same condition are indistinguishable. $W$ is sometimes called the "thermodynamic probability" since it is an integer greater than one, while mathematical probabilities are always numbers between zero and one.

Introduction of the natural logarithm

In Boltzmann’s 1877 paper, he clarifies molecular state counting to determine the state distribution number introducing the logarithm to simplify the equation.

Boltzmann writes: “The first task is to determine the permutation number, previously designated by 𝒫 , for any state distribution. Denoting by J the sum of the permutations 𝒫 for all possible state distributions, the quotient 𝒫 /J is the state distribution’s probability, henceforth denoted by W. We would first like to calculate the permutations 𝒫 for the state distribution characterized by w₀ molecules with kinetic energy 0, w₁ molecules with kinetic energy ϵ, etc. …

“The most likely state distribution will be for those w₀, w₁ … values for which 𝒫 is a maximum or since the numerator is a constant, for which the denominator is a minimum. The values w₀, w₁ must simultaneously satisfy the two constraints (1) and (2). Since the denominator of 𝒫 is a product, it is easiest to determine the minimum of its logarithm, …”

Therefore, by making the denominator small, he maximizes the number of states. So to simplify the product of the factorials, he uses their natural logarithm to add them. This is the reason for the natural logarithm in Boltzmann’s entropy formula.^[6]

Generalization

Boltzmann's formula applies to microstates of a system, each possible microstate of which is presumed to be equally probable.

But in thermodynamics, the universe is divided into a system of interest, plus its surroundings; then the entropy of Boltzmann's microscopically specified system can be identified with the system entropy in classical thermodynamics. The microstates of such a thermodynamic system are not equally probable—for example, high energy microstates are less probable than low energy microstates for a thermodynamic system kept at a fixed temperature by allowing contact with a heat bath. For thermodynamic systems where microstates of the system may not have equal probabilities, the appropriate generalization, called the Gibbs entropy, is:

S_{\mathrm {G} }=-k_{\mathrm {B} }\sum p_{i}\ln p_{i}

3

This reduces to equation (1) if the probabilities p_i are all equal.

Boltzmann used a $\rho \ln \rho$ formula as early as 1866.^[7] He interpreted $ρ$ as a density in phase space—without mentioning probability—but since this satisfies the axiomatic definition of a probability measure we can retrospectively interpret it as a probability anyway. Gibbs gave an explicitly probabilistic interpretation in 1878.

Boltzmann himself used an expression equivalent to (3) in his later work^[8] and recognized it as more general than equation (1). That is, equation (1) is a corollary of equation (3)—and not vice versa. In every situation where equation (1) is valid, equation (3) is valid also—and not vice versa.

Boltzmann entropy excludes statistical dependencies

The term Boltzmann entropy is also sometimes used to indicate entropies calculated based on the approximation that the overall probability can be factored into an identical separate term for each particle—i.e., assuming each particle has an identical independent probability distribution, and ignoring interactions and correlations between the particles. This is exact for an ideal gas of identical particles that move independently apart from instantaneous collisions, and is an approximation, possibly a poor one, for other systems.^[9]

The Boltzmann entropy is obtained if one assumes one can treat all the component particles of a thermodynamic system as statistically independent. The probability distribution of the system as a whole then factorises into the product of N separate identical terms, one term for each particle; and when the summation is taken over each possible state in the 6-dimensional phase space of a single particle (rather than the 6N-dimensional phase space of the system as a whole), the Gibbs entropy

S_{\mathrm {G} }=-Nk_{\mathrm {B} }\sum _{i}p_{i}\ln p_{i}

4

simplifies to the Boltzmann entropy $S_{\mathrm {B} }$ .

This reflects the original statistical entropy function introduced by Ludwig Boltzmann in 1872. For the special case of an ideal gas it exactly corresponds to the proper thermodynamic entropy.

For anything but the most dilute of real gases, $S_{\mathrm {B} }$ leads to increasingly wrong predictions of entropies and physical behaviours, by ignoring the interactions and correlations between different molecules. Instead one must consider the ensemble of states of the system as a whole, called by Boltzmann a holode, rather than single particle states.^[10] Gibbs considered several such kinds of ensembles; relevant here is the canonical one.^[9]

References

^ See: photo of Boltzmann's grave in the Zentralfriedhof, Vienna, with bust and entropy formula.
^ Boltzmann equation. Eric Weisstein's World of Physics (states the year was 1872).
^ Perrot, Pierre (1998). A to Z of Thermodynamics. Oxford University Press. ISBN 0-19-856552-6. (states the year was 1875)
^ Max Planck (1914) The theory of heat radiation equation 164, p.119
^ Cercignani, C. (1998). Ludwig Boltzmann: the Man who Trusted Atoms, Oxford University Press, Oxford UK, ISBN 9780198501541, p. 134, pp. 141–142.
^ Sharp, K.; Matschinsky, F. Translation of Ludwig Boltzmann’s Paper “On the Relationship between the Second Fundamental Theorem of the Mechanical Theory of Heat and Probability Calculations Regarding the Conditions for Thermal Equilibrium” Sitzungberichte der Kaiserlichen Akademie der Wissenschaften. Mathematisch-Naturwissen Classe. Abt. II, LXXVI 1877, pp 373-435 (Wien. Ber. 1877, 76:373-435). Reprinted in Wiss. Abhandlungen, Vol. II, reprint 42, p. 164-223, Barth, Leipzig, 1909. Entropy 2015, 17, 1971-2009. https://doi.org/10.3390/e17041971 This article incorporates text from this source, which is available under the CC BY 3.0 license.
^ Ludwig Boltzmann (1866). "Über die Mechanische Bedeutung des Zweiten Hauptsatzes der Wärmetheorie". Wiener Berichte. 53: 195–220.
^ Ludwig Boltzmann (1896). Vorlesungen über Gastheorie, vol. I. J.A. Barth, Leipzig.; Ludwig Boltzmann (1898). Vorlesungen über Gastheorie, vol. II. J.A. Barth, Leipzig.
^ ^a ^b Jaynes, E. T. (1965). Gibbs vs Boltzmann entropies. American Journal of Physics, 33, 391-8.
^ Cercignani, C. (1998). Ludwig Boltzmann: the Man who Trusted Atoms, Oxford University Press, Oxford UK, ISBN 9780198501541, p. 134.

External links

[1] See: photo of Boltzmann's grave in the Zentralfriedhof, Vienna, with bust and entropy formula.

[2] Boltzmann equation. Eric Weisstein's World of Physics (states the year was 1872).

[3] Perrot, Pierre (1998). A to Z of Thermodynamics. Oxford University Press. ISBN 0-19-856552-6. (states the year was 1875)

[boltzmannEntropy-4] Max Planck (1914) The theory of heat radiation equation 164, p.119

[5] Cercignani, C. (1998). Ludwig Boltzmann: the Man who Trusted Atoms, Oxford University Press, Oxford UK, ISBN 9780198501541, p. 134, pp. 141–142.

[6] Sharp, K.; Matschinsky, F. Translation of Ludwig Boltzmann’s Paper “On the Relationship between the Second Fundamental Theorem of the Mechanical Theory of Heat and Probability Calculations Regarding the Conditions for Thermal Equilibrium” Sitzungberichte der Kaiserlichen Akademie der Wissenschaften. Mathematisch-Naturwissen Classe. Abt. II, LXXVI 1877, pp 373-435 (Wien. Ber. 1877, 76:373-435). Reprinted in Wiss. Abhandlungen, Vol. II, reprint 42, p. 164-223, Barth, Leipzig, 1909. Entropy 2015, 17, 1971-2009. https://doi.org/10.3390/e17041971 This article incorporates text from this source, which is available under the CC BY 3.0 license.

[wt-7] Ludwig Boltzmann (1866). "Über die Mechanische Bedeutung des Zweiten Hauptsatzes der Wärmetheorie". Wiener Berichte. 53: 195–220.

[vorlesungen-8] Ludwig Boltzmann (1896). Vorlesungen über Gastheorie, vol. I. J.A. Barth, Leipzig.; Ludwig Boltzmann (1898). Vorlesungen über Gastheorie, vol. II. J.A. Barth, Leipzig.

[JaynesBoltzmannGibbs-9] Jaynes, E. T. (1965). Gibbs vs Boltzmann entropies. American Journal of Physics, 33, 391-8.

[10] Cercignani, C. (1998). Ludwig Boltzmann: the Man who Trusted Atoms, Oxford University Press, Oxford UK, ISBN 9780198501541, p. 134.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

v t e Statistical mechanics
Theory	Principle of maximum entropy ergodic theory
Statistical thermodynamics	Ensembles partition functions equations of state thermodynamic potential: U H F G Maxwell relations
Models	Ferromagnetism models Ising Potts Heisenberg percolation Particles with force field depletion force Lennard-Jones potential
Mathematical approaches	Boltzmann equation H-theorem Vlasov equation BBGKY hierarchy stochastic process mean-field theory and conformal field theory
Critical phenomena	Phase transition Critical exponents correlation length size scaling
Entropy	Boltzmann Shannon Tsallis Rényi von Neumann
Applications	Statistical field theory elementary particle superfluidity Condensed matter physics Complex system chaos information theory Boltzmann machine