Thursday, June 28, 2012
Maximum Entropy Distributions



Entropy is an important topic in many fields; it has very well known uses in statistical mechanics, thermodynamics, and information theory. The classical formula for entropy is Σi(pi log pi), where p=p(x) is a probability density function describing the likelihood of a possible microstate of the system, i, being assumed. But what is this probability density function? How must the likelihood of states be configured so that we observe the appropriate macrostates?



In accordance with the second law of thermodynamics, we wish for the entropy to be maximized. If we take the entropy in the limit of large N, we can treat it with calculus as S[φ]=∫dx φ ln φ. Here, S is called a functional (which is, essentially, a function that takes another function as its argument). How can we maximize S? We will proceed using the methods of calculus of variations and Lagrange multipliers.



First we introduce three constraints. We require normalization, so that ∫dx φ = 1. This is a condition that any probability distribution must satisfy, so that the total probability over the domain of possible values is unity (since we’re asking for the probability of any possible event occurring). We require symmetry, so that the expected value of x is zero (it is equally likely to be in microstates to the left of the mean as it is to be in microstates to the right — note that this derivation is treating the one-dimensional case for simplicity). Then our constraint is ∫dx x·φ = 0. Finally, we will explicitly declare our variance to be σ², so that ∫dx x²·φ = σ².



Using Lagrange multipliers, we will instead maximize the augmented functional S[φ]=∫(φ ln φ + λ0φ + λ1xφ + λ2x²φ dx). Here, the integrand is just the sum of the integrands above, adjusted by Lagrange multipliers λk for which we’ll be solving.



Applying the Euler-Lagrange equations and solving for φ gives φ = 1/exp(1+λ0+xλ1+x²λ2). From here, our symmetry condition forces λ1=0, and evaluating the other integral conditions gives our other λ’s such that q = (1/2πσ²)½·exp(-x² / 2σ²), which is just the Normal (or Gaussian) distribution with mean 0 and variance σ². This remarkable distribution appears in many descriptions of nature, in no small part due to the Central Limit Theorem.

Maximum Entropy Distributions

Entropy is an important topic in many fields; it has very well known uses in statistical mechanics, thermodynamics, and information theory. The classical formula for entropy is Σi(pi log pi), where p=p(x) is a probability density function describing the likelihood of a possible microstate of the system, i, being assumed. But what is this probability density function? How must the likelihood of states be configured so that we observe the appropriate macrostates?

In accordance with the second law of thermodynamics, we wish for the entropy to be maximized. If we take the entropy in the limit of large N, we can treat it with calculus as S[φ]=∫dx φ ln φ. Here, S is called a functional (which is, essentially, a function that takes another function as its argument). How can we maximize S? We will proceed using the methods of calculus of variations and Lagrange multipliers.

First we introduce three constraints. We require normalization, so that ∫dx φ = 1. This is a condition that any probability distribution must satisfy, so that the total probability over the domain of possible values is unity (since we’re asking for the probability of any possible event occurring). We require symmetry, so that the expected value of x is zero (it is equally likely to be in microstates to the left of the mean as it is to be in microstates to the right — note that this derivation is treating the one-dimensional case for simplicity). Then our constraint is ∫dx x·φ = 0. Finally, we will explicitly declare our variance to be σ², so that ∫dx x²·φ = σ².

Using Lagrange multipliers, we will instead maximize the augmented functional S[φ]=∫(φ ln φ + λ0φ + λ1xφ + λ2x²φ dx). Here, the integrand is just the sum of the integrands above, adjusted by Lagrange multipliers λk for which we’ll be solving.

Applying the Euler-Lagrange equations and solving for φ gives φ = 1/exp(1+λ0+xλ1+x²λ2). From here, our symmetry condition forces λ1=0, and evaluating the other integral conditions gives our other λ’s such that q = (1/2πσ²)½·exp(-x² / 2σ²), which is just the Normal (or Gaussian) distribution with mean 0 and variance σ². This remarkable distribution appears in many descriptions of nature, in no small part due to the Central Limit Theorem.

Friday, January 13, 2012
Charge, Parity and Time Reversal (CPT) Symmetry
 From our everyday experience, it is easy to conclude that nature obeys the laws of physics with absolute consistency. However, several experiments have revealed certain cases where these laws are not the same for all particles and their antiparticles. The concept of a symmetry, in physics, means that the laws will be the same for certain types of matter. Essentially, there are three different kinds of known symmetries that exist in the universe: charge (C), parity (P), and time reversal (T). The violations of these symmetries can cause nature to behave differently. If C symmetry is violated, then the laws of physics are not the same for particles and their antiparticles. P symmetry violation implies that the laws of physics are different for particles and their mirror images (meaning the ones that spin in the opposite direction). The violation of symmetry T indicates that if you go back in time, the laws governing the particles change.
 There were two American physicists by the names of Tsunng-Dao Lee and Chen Ning Yang suggested that the weak interaction violates P symmetry.  This was proven by an experiment which was conducted with radioactive atoms of colbalt-60 that were lined up and introduced a magnetic field to insure that they are spinning in the same direction. In addition, it was also found that the weak force also does not obey symmetry C. Oddly enough, the weak force did appear to obey the combined CP symmetry. Therefore the laws of physics would be the same for a particle and it’s antiparticle with opposite spin.
Surprise, surprise! There was a slight error in the previous experiment that was just mentioned. A few years later, it was discovered that the weak force actually violates CP symmetry. Another experiment was conducted by two physicists named Cronin and Fitch. They studied the decay of neutral kaons, which are mesons that are composed of either one down quark (or antiquark) and a strange antiquark (or quark). These particles have two decay modes where one will decay much faster than the other, even though they all have identical masses. The particles with the longer lifetimes will decay into three pions (denoted with the symbol π0), however the kaon ‘species’ with the shorter lifetimes will only decay into two pions. They had a 57 foot beamline, where they only expected to see the particles with slower decay rate at the end of the beam tube. In astonishment, one out of every 500 decays where from the kaons species that had a shorter lifetime. The main conflict with seeing the short-lived mesons at the end of the beam tube is because they are traveling relavistic speeds and therefore ignoring the time dilatationthat they are supposed to undergo. Thus, the experiment has shown that the weak force causes a small CP violation that can be seen in kaon decay.

Charge, Parity and Time Reversal (CPT) Symmetry

From our everyday experience, it is easy to conclude that nature obeys the laws of physics with absolute consistency. However, several experiments have revealed certain cases where these laws are not the same for all particles and their antiparticles. The concept of a symmetry, in physics, means that the laws will be the same for certain types of matter. Essentially, there are three different kinds of known symmetries that exist in the universe: charge (C), parity (P), and time reversal (T). The violations of these symmetries can cause nature to behave differently. If C symmetry is violated, then the laws of physics are not the same for particles and their antiparticles. P symmetry violation implies that the laws of physics are different for particles and their mirror images (meaning the ones that spin in the opposite direction). The violation of symmetry T indicates that if you go back in time, the laws governing the particles change.

There were two American physicists by the names of Tsunng-Dao Lee and Chen Ning Yang suggested that the weak interaction violates P symmetry. This was proven by an experiment which was conducted with radioactive atoms of colbalt-60 that were lined up and introduced a magnetic field to insure that they are spinning in the same direction. In addition, it was also found that the weak force also does not obey symmetry C. Oddly enough, the weak force did appear to obey the combined CP symmetry. Therefore the laws of physics would be the same for a particle and it’s antiparticle with opposite spin.

Surprise, surprise! There was a slight error in the previous experiment that was just mentioned. A few years later, it was discovered that the weak force actually violates CP symmetry. Another experiment was conducted by two physicists named Cronin and Fitch. They studied the decay of neutral kaons, which are mesons that are composed of either one down quark (or antiquark) and a strange antiquark (or quark). These particles have two decay modes where one will decay much faster than the other, even though they all have identical masses. The particles with the longer lifetimes will decay into three pions (denoted with the symbol π0), however the kaon ‘species’ with the shorter lifetimes will only decay into two pions. They had a 57 foot beamline, where they only expected to see the particles with slower decay rate at the end of the beam tube. In astonishment, one out of every 500 decays where from the kaons species that had a shorter lifetime. The main conflict with seeing the short-lived mesons at the end of the beam tube is because they are traveling relavistic speeds and therefore ignoring the time dilatationthat they are supposed to undergo. Thus, the experiment has shown that the weak force causes a small CP violation that can be seen in kaon decay.

(Source: aps.org)