Boltzmann’s entropy formula is possibly the one of the most difficult equations in Physics. Not because the equation itself is that confusing(it isn’t, it is just two variables one constant and a trig function), but because it relates two things, entropy and probability of being indifferent energy states, that are both difficult to really understand.
So, as I usually do, I looked into the history of this law, who made it and why was it made? And I found that it was created by a scientist named Max Planck in 1900 and written almost identically to the modern form in 1901.
But wait, why is it called the Boltzmann equation and why is the constant in it Boltzmann’s constant and why is it even on Boltzmann’s freaking gravestone if Planck created it?
Table of Contents
Entropy by Rudolf Clausius
The Relationship of Statistics and Entropy
Max Planck
Planck’s Equation
Boltzmann’s Famous Equation
References
Entropy by Rudolf Clausius
I would like to start with the origin of the idea of entropy. In 1854, a German scientist named Rudolf Clausius noted that absorbing less heat at a lower temperature was equivalent to absorbing more heat at a higher temperature so he called the heat over the temperature the “equivalence value” (he would later call the equivalence value the entropy).
By 1862, Clausius found that an increase of the equivalence values of a single process would increase the separation of the molecules or disorganize their relationship. He also found that any decrease in “equivalence function” on one object would necessitate an increase of equal or greater value in other objects.
In other words, the “equivalence value” (entropy) of a closed system can only increase. Finally, in 1865, Clausius renamed the “equivalence value” the entropy and gave it the letter S and defined the second law of thermodynamics to be the entropy of the universe can only increase, which is a modern definition.
Now I want to take a moment to talk about the history of probability in thermodynamics and I am going to also start with… ready for it… a German man named Rudolf Clausius! In 1857, three years after Clausius’s first paper introducing the “equivalence value”,
Clausius wrote a paper on what the temperature meant about the motion of molecules[1]. In this paper Clausius became the first person to include rotational and vibrational motion of molecules as well as linear motion. Even with these limitations, Clausius found that molecules move at very fast speeds.
For example, Clausius found that hydrogen gas at 0 Celsius should move at slightly more than5 times the speed of sound! After reading Clausius another scientist published an objection, if gas molecules are really moving so fast how come cigar smoke doesn’t fill the room faster than the speed of sound?
Clausius felt like this was an interesting objection (writing he “rejoice[s] at the discussion[2]”), but instead of invalidating his theories Clausius decided that it would all work out if the molecules are moving very fast but not very far.
In other words, in cigar smoke (or in any gas) there are a ridiculous number of molecules moving in all directions but they don’t get very far before bouncing off of another molecule and changing direction, so that even though the individual gas molecules are moving very quickly, the gas itself diffuses slowly.
He then came up with a term he called the “mean length of the path” which was “how far on an average can the molecule move, before its center of gravity comes into the sphere of action of another molecule.[3]”Conveniently an English scientist named Frederick Guthrie was a fan of Clausius and was also bilingual (he received his Ph.D. in Germany with one of my favorite scientists Robert Bunsen) and translated Clausius’s work into English and published it in February of 1859
Three months later a 27-year-old Scottish scientist named James Clerk Maxwell wrote a friend that Clausius’s paper had inspired him to determine the mean path by, “comparing it with phenomena which seem to depend on this “mean path” …[like] internal friction of gases, diffusion of gases, and conduction of heat through a gas[4]”.
Between 1860 and 1866, Maxwell produced a series of papers on what he called “the Dynamical Theory of Gases” although he admitted it was, “Professor Clausius, to whom we owe the most extensive developments of the dynamical theory of gases.[5]”
During the same time, the busy Maxwell also published a couple of papers on equations of electricity and magnetism the results of which are known as Maxwell’s laws! Funny story, in 1861, Maxwell was once stuck in a crowd and Michael Faraday saw him and, referring to his work in statistics, shouted, “Ho, Maxwell, cannot you get out? If any man can find his way through a crowd it should be you.[6]”
Clausius had some minor issues with Maxwell’s theories but there was another German scientist who was entranced, and his name was Ludwig Boltzmann (yes, he is involved).
Boltzmann was 13 years younger than Maxwell and went to college in 1863. While still an undergraduate one of his professors instantly realized his brilliance and, according to Boltzmann, he immediately decided, “to hand me a copy of Maxwell’s papers [on gases]” however, “at that time I did not understand a word of English, [so] he also gave me an English grammar [book].
[7]” Boltzmann managed to translate Maxwell and started to publish his own papers on gasses. Three years later, in 1866, Boltzmann had earned his Ph.D. in the kinetic theory of gasses and by 1869 25-year-old Boltzmann was made a full professor! Maxwell and Boltzmann’s work on statistics in thermodynamics led to the Maxwell-Boltzmann distribution, or equations for the probability of different speeds of gasses that is still used today.
Aside from his eerie intelligence Boltzmann was also known for his excellent teaching prowess, (Lisa Meitner described him as the best teacher she had ever had), his humor, (like when he said he was “tolerably” good at mathematics except “when I am counting beer glasses”), and his oddly naive view of reality (like when he consulted a professor of zoology to determine how to milk a cow).
Maxwell started being confused about entropy, however, once Maxwell’s confusion was cleared up (by an American named Josiah Gibbs), Maxwell started to combine his theories of using statistics to study molecules with entropy.
This led Maxwell to decide that, “the truth of the second law is therefore a statistical, not a mathematical truth, for it depends on the fact that the bodies we deal with consist of millions of molecules, and that we never can get hold of single molecules.” Or, as Maxwell amusingly put it to a friend, “The second law of Thermodynamics has the same degree of truth as the statement that if you throw a cup of water into the sea, you cannot get the same cup of water out again.
[8]”Not surprisingly, Clausius was not pleased with Maxwell for downgrading “his” second law like that. However, Clausius had other issues: first in 1870, Clausius had been injured in the Franco-Prussian war when he organized an ambulance corp. Second, in 1875, Clausius’s wife died in childbirth and Clausius took time off of his research to focus on raising his six children and teaching (Clausius always taught, even on his death bed).
The Relationship of Statistics and Entropy
Young Boltzmann, however, was motivated once again by Maxwell and now dived into the relationship between statistics and entropy. In 1872, Boltzmann wrote that, “the molecules of [a] body are indeed so numerous, and their motion is so rapid, that we can perceive nothing more than average values…
Hence, the problems of the mechanical theory of heat are also problems of probability theory.[9]”By 1877, Boltzmann started to work out the relationship between probability and entropy. He started with the idea that “in the game of Lotto any individual set of five numbers is just as improbable as the set 1,2,3,4,5.
It is only because there are many more uniform distributions than non-uniform ones that the distribution of states will become uniform in the course of time.[10]” In other words, entropy increases because there are “infinitely many more uniform than non-uniform distributions[11]”.
Boltzmann even declared that one “could even calculate, from the relative numbers of the different state distributions, their probabilities[12].” So, in October of 1877 Boltzmann set out to find the actual equation for the different state distributions to determine the entropy from the number of arrangements of molecules.
He ended up writing over 50 pages of dense, equation rich material where he formulated how one could determine the probability of having molecules in different states. Boltzmann broke up the energies of molecules into different sections and predicted the probability of ending up with different scenarios.
He wrote, “The initial state will, in most cases, be a very unlikely one, and the system will move from it to more and more probable states until at last it becomes the most probable, so that the heat equilibrium has reached. Applying this to the second law, we can identify the quantity which is usually called the entropy, with the probability of the condition in question.[13]”However, Boltzmann did not make an equation between entropy and his newly defined probability.
Fascinatingly, both Maxwell and Boltzmann might have had the assistance of their scientific wives. Katherine Maxwell had no scientific or mathematical training but it was recorded that she helped her husband with his experiments.
In fact, in 1877, a friend asked Maxwell for some data about viscosity and Maxwell wrote back, “my better 1/2, who did all the real work of the kinetic theory is at present engaged in other researches. When she is done, I will let you know the answer to your enquiry.[14]” Boltzmann’s wife, Henriette Boltzmann, actually tried to study Physics and mathematics at school and even basically sued to be able to audit classes on Boltzmann’s suggestion.
In Boltzmann’s marriage proposal he referred to her as a “mathematician” and wrote, “it seems to me that permanent love cannot exist if [a wife] has no understanding and enthusiasm for her husband’s efforts, and is just his maid and not the companion who struggles alongside him.[15]”
However, she was probably pretty busy raising their five children and managing Boltzmann’s depressed moods. The Boltzmann’s and the Maxwell’s built on each other’s ideas for a few years but then James Clerk Maxwell fell ill in spring of 1877 and he died of stomach cancer when he was only 48 years old in November of 1879.
Henriette and Ludwig Boltzmann continued to publish their theories of heat, probability, and motion. However, they felt increasingly outnumbered in Germany as the existence of molecules was considered an avant-garde idea that was deeply unpopular among the scientific establishment.
Max Planck
Now we finally get to Max Planck. Planck was 14 years younger than Boltzmann and in 1877, the same year that Maxwell fell ill and Boltzmann wrote his very long paper on probability, Planck went to graduate school and found to his disappointment that the great researchers there were pretty awful teachers.
He decided he was better off just reading original works. Years later he recalled, “One day, I happened to come across the treatises of Rudolf Clausius, whose lucid style and enlightening clarity of reasoning made an enormous impression on me, and I became deeply absorbed in his articles, with an ever-increasing enthusiasm.
I appreciated especially his exact formulation of the two laws of thermodynamics[16]”. By 1879, Planck got his Ph.D. on the second law of thermodynamics however, “The effect of my dissertation on the physicists of those days was nil. None of my professors at the University had any understanding for its contents…. I found no interest, let alone approval, even among the very physicists who were closely concerned with the topic.
[17]” Even Clausius was indifferent although Planck tried to visit Clausius at home to no reply. Despite that, Planck managed to become a professor in Berlin in 1889. Planck then got into a debate with the “energetics” or scientists who believed in energy but not entropy and who also disagreed with an absolute temperature scale and did not believe in the existence of atoms.
Planck found a strange defender, Ludwig Boltzmann. In 1896, Boltzmann had a public debate with an “energetic” named Ostwald on the issue which seemed to convince at least the younger generation of the validity of the second law but Planck felt that Boltzmann was also attacking Planck.
See, Planck did not like Boltzmann’s probability theories, recalling “I regarded the principle of the increase of entropy as no less immutably valid than the principle of the conservation of energy itself, whereas Boltzmann treated the former merely as a low of probabilities[18]” and for that reason “I was not only indifferent but to a certain extent even hostile to the atomic theory which was the foundation of his entire research.[19]”
Planck’s Equation
Meanwhile, Planck had moved on to a new subject that would, eventually, force him to accept Boltzmann’s method of probabilities. See, in 1894, an acquittance of Planck’s named Wilhelm Wien had made an equation to describe the distribution of radiation from a perfectly absorptive object called a “black body” that was based on experimental data but had no theoretical backing.
Planck felt that he had a unique view on the problem as no one else worried about entropy, recalling, “it was an odd jest of fate that…. The lack of interest of my colleagues in [entropy] now turned out to be an outright boon.[20]”By 1899, Planck published his results and for a short while it was called the Plank-Wien law.
However, there was a problem with the law, it didn’t work at low energies and in October of 1900, Planck was told by an experimentalist that he needed to tweak his equation. Without any time to prove anything, Planck made up a new equation that would both work at low frequencies and also would look like the Plank-Wien law at higher frequencies.
The experimentalists were delighted but Planck was distraught, a theoretician is not supposed to just guess an equation from the data, he or she is supposed to derive it from fundamental theorems.So, Planck went to work. Planck recalled, “For six years I had struggled with the blackbody theory. I knew the problem was fundamental and I knew the answer.
I had to find a theoretical explanation at any cost, except the inviolability of the two laws of thermodynamics[21].” So, in a self-described “act of desperation” Planck turned to Boltzmann’s method of probability. After a few months of intensive study Planck decided that, “Since the entropy S is an additive magnitude but the probability W is a multiplicative one, I simply postulated that S = k log W, where k is a universal constant.
[22]” [A quick note about the log function, it is the inverse of the exponent function, so that if you multiply the input by 10, the value of the log (base 10) increases by 1. This is what Planck meant about additive and multiplicative.]
Boltzmann’s Famous Equation
Here is Boltzmann’s famous equation but what does it mean? First, it means that entropy has an absolute value that one can calculate from the properties of the molecules in a material. Although not as easy to measure as the temperature, the entropy is a fundamental feature of a material with a definite value
. The more arrangements of molecules you have, the more entropy you have. Second, the constant, k, has meaning above and beyond just a relationship between the entropy and probability. It is used in many different equations including the heat capacity and the relationship between the kinetic energy of particles and the temperature.
In fact, k is a link between the microscopic and the macroscopic. With an accurate measure of k, one could predict many things that wasn’t known accurately at the time, including the mass of a hydrogen atom and from that, the charge of a single electron. In fact, as of 2019, k, is one of only seven basic constants (like the speed of light) that all of our units of everything is currently based on!
But then Planck had a problem. If the energy is continuous, then the probability and thus the entropy would be infinite, or as Planck put it, if the energy is, “considered to be continuously divisible quantity, this distribution is possible in infinitely many ways.
[23]” For that reason, Planck constrained the energy to be created in little energy packets with energy equal to a constant, h, times the frequency. This is the origin of quantum mechanics(so you could argue that Boltzmann’s entropy equation also led to the formation of quantum theory).
In addition, h is another one of the seven fundamental constants that we base our units on! Then, with the voluminous data of blackbody radiation, Planck managed to get values for his two constants, h and k. Startingly, h was around 1% of its current value and k was 2.5% off from its current value.
Even more impressively, with this data, Planck managed to get a value for the charge on an electron within 2% of its current value (when the other two accepted values at the time were 73% too low or 35% too high).
Now you might ask how this equation (and the constant) were named after Boltzmann and how that ended up on his grave. Planck was largely to blame for this.
First, when he introduced his entropy equation he was following a new method for him and therefore said he was, “introducing probability considerations, the importance of which for the second law of thermodynamics was first of all discovered by Mr. Boltzmann,[24]” which implied to the people who hadn’t poured through Boltzmann’s 50 page paper that Boltzmann had made the equation.
Then, in September of 1906, Boltzmann’s depression got the better of him and he committed suicide. At the same time, Planck was giving a series of lectures on the “Theory of Heat Radiation” and learned through some of the scientific memorials at Boltzmann’s funeral that Boltzmann had, in 1896 (4 years before Planck wrote his equation), wrote that a function that was related to the entropy was also related to the log of the probability[25]
. For that reason, Planck noted in his lectures that “the logarithmic connection between entropy and probability was first stated by Boltzmann in his kinetic theory of gasses [in 1896][26]” and then added that it didn’t really count because Boltzmann never had a constant, and as Boltzmann never directly linked entropy to the log of probability, Boltzmann had created an unspecified value for the entropy.
A few years later, Planck published these lectures as a book which ended up having the opposite of the desired effect as that one quote is *still to this day* used to justify the naming of the equation that Planck created to Boltzmann! Also, during the 1910s, it became popular to name constants after the scientist who discovered them.
For example, in July of 1913, Niels Bohr wrote his famous paper about “the constitution of atoms” where he named the constant h, in e = hf, “Planck’s constant[27]” (before it was mostly called “the elementary quantum of action[28]”.) Bohr’s paper was a hit, years later Einstein recalled that Bohr’s paper, “appeared to me like a miracle – and appears to me as a miracle even today. This is the highest form of musicality in the sphere of thought.
[29]” Max Planck opined that Bohr discovered, “the long-sought key to the entrance gate into the wonderland of spectroscopy…. And now that way was opened, a sudden flood of new-won knowledge poured over the whole field including the neighboring fields in physics and chemistry.
[30]” Six months later, English scientists Ernest and Ezer Griffiths published a detailed paper on the specific heat of metals at very low temperatures and as far as I can tell, became the first people to call k “Boltzmann’s constant.
[31]” This paper was not revolutionary like Bohr’s but it included a lot of impressive data on an important subject as well as a good overview of the quantum view of solids at the time, so it was widely read. Soonmost “modern” physicists were discussing quantum effects and using “Planck’s constant” and “Boltzmann’s constant” to do so.
By 1920 when Planck gave a speech accepting his Nobel Prize he noted that k was “often referred to as Boltzmann’s constant” which he found to be “a peculiar state of affairs” as “to my knowledge, Boltzmann himself never introduced it.
[32]” By the 1930s, it became trendy to name equations after their “inventors” and soon e = hf was commonly called “Planck’s equation” *because* of Planck’s constant and “S = k Log W” was “Boltzmann’s entropy equation” *because* of Boltzmann’s constant and the names have stuck to this day.
Finally, in 1935 the alderman of Vienna realized that Boltzmann’s tomb was decrepit and didn’t honor their native son very well. He therefore decided that “the City of Vienna and its people are proud of the man of genius who lived here, and they have endeavored to find a burial place worthy of him[33]” and for that reason they moved his body to a place of honor and created a white marble bust of Boltzmann and a local scientist named Hans Thirring had them add “Boltzmann’s greatest scientific achievement[34]” to his grave, ie. “S = k log W”.
So that is how “Boltzmann’s equation” ended up on his grave even though it should probably be called “Planck’s entropy equation” and “Boltzmann’s constant” should definitely be called “Planck’s (other) Constant”. Now I am a bit of an impasse as there are several things I could do for my next video.
So, I thought I would do something I have never done before… let you choose what I should do for my next video. I have already made videos about the first three laws of thermodynamics and more about the influence of Planck’s quantum equation, and one about Maxwell’s laws but what should I do next?
Should I make a video about the heroic and tragic life of Max Planck? Or should I make a video about Niels Bohr’s “musical” quantum model of nature that we still study even though it is wrong? Or should I go back in time and talk about an amazing woman who I missed, a scientist in the 1700s who brought electricity to Italy and whose marriage was considered a travesty? Or should I skip all this theory and jump up to the wild story of the teenager who invented television?
References
[1] Clausius, R “On the Nature of the Motion which we call heat” The London, Edinburgh and Dublin Phil. Magazine Ser. 4 Vol 14 (1857) p. 108
[2] “I rejoice at the discussion of this point by M. Buijs-Ballot, inasmuch as it affords me a desired opportunity of completing this part of my theory (which was perhaps discussed too briefly in my paper), and to prevent thereby further misunderstandings.” Clausius, R “On the Mean Length of the Paths…” The London, Edinburgh and Dublin Phil. Magazine Ser. 4 Vol 17 (Feb 1859) p. 81-2
[3] Clausius, R “On the Mean Length of the Paths…” The London, Edinburgh and Dublin Phil. Magazine Ser. 4 Vol 17 (Feb 1859) p. 84
[4] James Clerk Maxwell to George Stokes (May 30, 1859) Memoir and Scientific Correspondence of the Late Sir George Gabriel Stokes (1907) p. 8
[5] Maxwell, J “The Dynamical Theory of Gases” (May, 1866) The Scientific Papers of James Clerk Maxwell (1890) p. 28
[6] Faraday to Maxwell according to Campbell, L The Life of James Clerk Maxwell (1882) p. 319
[7] Ludwig Boltzmann obituary of Jose Stefan translated and found in Cercignani, C Ludwig Boltzmann: The Man Who Trusted Atoms (2006) p.
[8] recalled on p 55 “Einstein and the Quantum” Stone
[9] Boltzmann, L “Further Studies on the Thermal Equilibrium of Gas Molecules” (1872) translated and found in Brush, S The Kinetic Theory of Gases (2003) p. 264
[10] Boltzmann, L “On the Relation of a General Mechanical Theorem to the Second Law of Thermodynamics” (1877) translated and found in Brush, S The Kinetic Theory of Gases (2003) p. 366
[11] Boltzmann, L “On the Relation of a General Mechanical Theorem to the Second Law of Thermodynamics” (1877) translated and found in Brush, S The Kinetic Theory of Gases (2003) p. 366
[12] Boltzmann, L “On the Relation of a General Mechanical Theorem to the Second Law of Thermodynamics” (1877) translated and found in Brush, S The Kinetic Theory of Gases (2003) p. 366
[13] Boltzmann, L “On the Relationship between the Second Theorem of the Mechanical Theory and the Probability Theory” Sitzungsberichte der Mathematisch-Naturwissenschaftlichen v. 76 (1877) p. 374
[14] James Clerk Maxwell to Peter Tait (Dec 29, 1877) found in Maxwell, J The Scientific Letters and Papers of James Clerk Maxwell, Volume 3 p. 559
[15] Ludwig Boltzmann to Henrietta von Aigentler (Sept 27, 1875) translated and found in Cercignani, C Ludwig Boltzmann: The Man Who Trusted Atoms (2006) p. 125
[16] Planck, M Scientific Autobiography, and other papers p. 17
[17] Planck, M Scientific Autobiography, and other papers p. 17
[18] Planck, M Scientific Autobiography, and other papers p. 25
[19] Planck, M Scientific Autobiography, and other papers p. 24
[20] Planck, M Scientific Autobiography, and other papers p. 24
[21] 1931 letter to R. Wood recounted in p 76 “From X-rays to Quarks: Modern Physicists and Their Discoveries” Emilio Segre.
[22] Planck, M Scientific Autobiography and Other Papers (1947) p. 41
[23] Planck, M “On the Theory of the Energy Distribution Law of the Normal Spectrum” English translation by D. HaarThe Old Quantum Theory (1967) p. 85
[24] Planck, M “On the Theory of the Energy Distribution Law of the Normal Spectrum” (1900) English translation from Haar, D. The Old Quantum Theory (1967) p. 83
[25] Boltzmann, L Lectures on Gas Theory (1896) p. 50
[26] Planck, Max The Theory of Heat Radiation English Translation, Blakiston (1914) p. 119
[27]Bohr, N “On The Constitution of Atoms and Molecules” London, Edin. Journal of Science Sixth Series (July 1913) p. 2
[28] Bohr, N “On The Constitution of Atoms and Molecules” London, Edin. Journal of Science Sixth Series (July 1913) p. 2
[29] Albert Einstein quoted in Stone, D Einstein and the Quantum (2015) p. 180
[30] Max Planck, “Nobel Prize Lecture 1920”
[31] Griffiths, E, and Griffiths, E “The Capacity for Heat of Metals at Low Temperatures” (June 25, 1914) Transactions of the Royal Society of London Vol. 214 (1914) p. 343
[32] Max Planck, “Nobel Prize Lecture 1920”
[33]Eftekhari, A “Ludwig Boltzmann” http://philsci-archive.pitt.edu/1717/2/Ludwig_Boltzmann.pdf p. 2
[34]Eftekhari, A “Ludwig Boltzmann” http://philsci-archive.pitt.edu/1717/2/Ludwig_Boltzmann.pdf p. 2