Thursday, June 28, 2012
Maximum Entropy Distributions



Entropy is an important topic in many fields; it has very well known uses in statistical mechanics, thermodynamics, and information theory. The classical formula for entropy is Σi(pi log pi), where p=p(x) is a probability density function describing the likelihood of a possible microstate of the system, i, being assumed. But what is this probability density function? How must the likelihood of states be configured so that we observe the appropriate macrostates?



In accordance with the second law of thermodynamics, we wish for the entropy to be maximized. If we take the entropy in the limit of large N, we can treat it with calculus as S[φ]=∫dx φ ln φ. Here, S is called a functional (which is, essentially, a function that takes another function as its argument). How can we maximize S? We will proceed using the methods of calculus of variations and Lagrange multipliers.



First we introduce three constraints. We require normalization, so that ∫dx φ = 1. This is a condition that any probability distribution must satisfy, so that the total probability over the domain of possible values is unity (since we’re asking for the probability of any possible event occurring). We require symmetry, so that the expected value of x is zero (it is equally likely to be in microstates to the left of the mean as it is to be in microstates to the right — note that this derivation is treating the one-dimensional case for simplicity). Then our constraint is ∫dx x·φ = 0. Finally, we will explicitly declare our variance to be σ², so that ∫dx x²·φ = σ².



Using Lagrange multipliers, we will instead maximize the augmented functional S[φ]=∫(φ ln φ + λ0φ + λ1xφ + λ2x²φ dx). Here, the integrand is just the sum of the integrands above, adjusted by Lagrange multipliers λk for which we’ll be solving.



Applying the Euler-Lagrange equations and solving for φ gives φ = 1/exp(1+λ0+xλ1+x²λ2). From here, our symmetry condition forces λ1=0, and evaluating the other integral conditions gives our other λ’s such that q = (1/2πσ²)½·exp(-x² / 2σ²), which is just the Normal (or Gaussian) distribution with mean 0 and variance σ². This remarkable distribution appears in many descriptions of nature, in no small part due to the Central Limit Theorem.

Maximum Entropy Distributions

Entropy is an important topic in many fields; it has very well known uses in statistical mechanics, thermodynamics, and information theory. The classical formula for entropy is Σi(pi log pi), where p=p(x) is a probability density function describing the likelihood of a possible microstate of the system, i, being assumed. But what is this probability density function? How must the likelihood of states be configured so that we observe the appropriate macrostates?

In accordance with the second law of thermodynamics, we wish for the entropy to be maximized. If we take the entropy in the limit of large N, we can treat it with calculus as S[φ]=∫dx φ ln φ. Here, S is called a functional (which is, essentially, a function that takes another function as its argument). How can we maximize S? We will proceed using the methods of calculus of variations and Lagrange multipliers.

First we introduce three constraints. We require normalization, so that ∫dx φ = 1. This is a condition that any probability distribution must satisfy, so that the total probability over the domain of possible values is unity (since we’re asking for the probability of any possible event occurring). We require symmetry, so that the expected value of x is zero (it is equally likely to be in microstates to the left of the mean as it is to be in microstates to the right — note that this derivation is treating the one-dimensional case for simplicity). Then our constraint is ∫dx x·φ = 0. Finally, we will explicitly declare our variance to be σ², so that ∫dx x²·φ = σ².

Using Lagrange multipliers, we will instead maximize the augmented functional S[φ]=∫(φ ln φ + λ0φ + λ1xφ + λ2x²φ dx). Here, the integrand is just the sum of the integrands above, adjusted by Lagrange multipliers λk for which we’ll be solving.

Applying the Euler-Lagrange equations and solving for φ gives φ = 1/exp(1+λ0+xλ1+x²λ2). From here, our symmetry condition forces λ1=0, and evaluating the other integral conditions gives our other λ’s such that q = (1/2πσ²)½·exp(-x² / 2σ²), which is just the Normal (or Gaussian) distribution with mean 0 and variance σ². This remarkable distribution appears in many descriptions of nature, in no small part due to the Central Limit Theorem.

Friday, October 14, 2011
Going superfluid!
A liquid goes superfluid when it suddenly loses all internal friction and gains near infinite thermal conductivity. The combination of zero viscosity but nonzero surface tension allows a superfluid to creep up walls and back down the outside to drip from the bottom of open containers, or to completely cover the inner surface of sealed containers. Lack of viscosity also allows a superfluid to leak through a surface that is porous to any degree, because the molecules can slip through even microscopic holes. Superfluids furthermore exhibit a thermo-mechanical effect where they flow from colder to warmer temperatures, exactly the opposite of heat flow as stated by the laws of thermodynamics! That implies the remarkable property of superfluids of carrying zero entropy. Because of this, a perpetual fountain can be set up by shining light on a superfluid bath just below a vertical open capillary tube, causing the fluid to shoot up through and beyond the tube until its  contact with the air causes it to cease being a superfluid and fall back  down into the bath, whereby it will cool back into the superfluid state  and repeat the process.
So how does superfluidity work, exactly?
Makings of a superfluid
Physicists first got the inkling of something stranger than the norm when, around 1940, they cooled liquid helium (specifically, the 4He isotope) down to 2.17 K and it started exhibiting the above-mentioned properties. Since the chemical makeup of the helium didn’t change (it was still helium), the transformation to a superfluid state is a physical change, a phase transition, just like ice melting into liquid water. Perhaps for cold matter researchers, this transition to a new phase of matter makes up for the fact that helium doesn’t solidify even at 0 K except under large pressure - whereas ALL other substances solidify above 10 K.
[Phase diagram of 4He, source]
Helium is truly the only substance that never solidifies under its own vapor pressure.
Instead, when the temperature reaches the transition or lambda point, quantum physics takes hold and a fraction of the liquid particles drop into the same ground-energy quantum state. They move in lock-step, behaving identically and never getting in each others’ way. Thus we come to see that superfluidity is a kind of Bose-Einstein condensation, the general phenomenon of a substance’s particles simultaneously occupying the lowest-energy quantum state.
Read more:”This Month in Physics History: Discovery of Superfluidity, January 1938”. APS News: January 2006
Based on a project by Barbara Bai, Frankie Chan, and Michele Silverstein at Cornell University.

Going superfluid!

A liquid goes superfluid when it suddenly loses all internal friction and gains near infinite thermal conductivity. The combination of zero viscosity but nonzero surface tension allows a superfluid to creep up walls and back down the outside to drip from the bottom of open containers, or to completely cover the inner surface of sealed containers. Lack of viscosity also allows a superfluid to leak through a surface that is porous to any degree, because the molecules can slip through even microscopic holes. Superfluids furthermore exhibit a thermo-mechanical effect where they flow from colder to warmer temperatures, exactly the opposite of heat flow as stated by the laws of thermodynamics! That implies the remarkable property of superfluids of carrying zero entropy. Because of this, a perpetual fountain can be set up by shining light on a superfluid bath just below a vertical open capillary tube, causing the fluid to shoot up through and beyond the tube until its contact with the air causes it to cease being a superfluid and fall back down into the bath, whereby it will cool back into the superfluid state and repeat the process.

So how does superfluidity work, exactly?

Makings of a superfluid

Physicists first got the inkling of something stranger than the norm when, around 1940, they cooled liquid helium (specifically, the 4He isotope) down to 2.17 K and it started exhibiting the above-mentioned properties. Since the chemical makeup of the helium didn’t change (it was still helium), the transformation to a superfluid state is a physical change, a phase transition, just like ice melting into liquid water. Perhaps for cold matter researchers, this transition to a new phase of matter makes up for the fact that helium doesn’t solidify even at 0 K except under large pressure - whereas ALL other substances solidify above 10 K.

[Phase diagram of 4He, source]

Helium is truly the only substance that never solidifies under its own vapor pressure.

Instead, when the temperature reaches the transition or lambda point, quantum physics takes hold and a fraction of the liquid particles drop into the same ground-energy quantum state. They move in lock-step, behaving identically and never getting in each others’ way. Thus we come to see that superfluidity is a kind of Bose-Einstein condensation, the general phenomenon of a substance’s particles simultaneously occupying the lowest-energy quantum state.

Read more:
This Month in Physics History: Discovery of Superfluidity, January 1938”. APS News: January 2006

Based on a project by Barbara Bai, Frankie Chan, and Michele Silverstein at Cornell University.