Information-theoretic entropy and thermodynamic entropy

There’s a long history of attempts to prove that information theoretic entropy is somehow connected to thermodynamic entropy. Most of these attempts have used Maxwell’s Demon or elaborate thought experiments involving single-molecule gases in boxes with removable walls and pistons and such. The likes of Charles Bennett have then gone on to spin out various implications of these thought experiments for reversible computing. I’m convinced by Earman and Norton’s arguments that these attempts are all misguided. Recently, Ladyman, Presnell and Short [preprint] made a new attempt to prove that information-theoretic entropy does apply to probabilistic mixtures of macrostates.

Ladyman et al consider a thermodynamic system C that is prepared in thermodynamic states or with equal probability. Suppose the states and have entropies and respectively. What entropy should we assign to C? Ladyman et al consider three possibilities:

1. We cannot assign a unique entropy to C since is either or but we do not know which. We can only proceed disjunctively by considering each possibility separately.
2. In our calculations we should set to equal the weighted average of and , namely .
3. In our calculations we should set to equal the weighted average of and plus an additional term that represents a contribution to the thermodynamic entropy due to the probability distribution itself. In this particular case, (where k is Boltzmann’s constant).

They argue that answer 1 would violate the Kelvin statement of the second law of thermodynamics. They put forward a revised version of the Kelvin law that would cover probabilistic processes of the sort considered above. They then use this revised law and a thought experiment to derive the expression for given in answer 3. The term in answer 3 is supposed to validate the connection between information-theoretic entropy and thermodynamics.

I have two problems with what they do:

1. Their revised version of the Kelvin law annihilates their objections to answer 1.
2. The thought experiment with which they derive the expression for could be altered so as to derive other expressions for that are not amenable to an information theoretic interpretation.

In this installment I’ll just elaborate on point 1. The second point will be explicated in a later post.

Ladyman et al’s dismissal of the disjunctive approach towards the entropy of C is built on a thought experiment using a single-molecule gas. Suppose one has an ideal gas consisting of a single molecule in a box, connected to a heat reservoir maintained at a constant temperature. The thought experiment proceeds as follows:

1. A partition is inserted reversibly into the middle of the box. The particle is now on one side of the partition or the other with equal probability.
2. A piston is pushed in isothermally from one side, say the left, up to a small distance from the partition.
3. The partition is removed and the gas is allowed to expand isothermally against the piston until the gas is returned to its original state.

Under the disjunctive approach, two things can happen. Either the particle is on the left side of the partition, so that the work done in pushing the piston in is very large (tending to infinity as the distance the piston stops from the partition gets smaller), or the particle is on the right side of the partition, so that no work is done in pushing the piston in, but the gas performs some work in pushing the piston back out. In the second scenario, a cyclic process occurs in which a net positive amount of heat is absorbed from the reservoir and the system does a net amount of work. This violates Lord Kelvin’s formulation of the second law of thermodynamics. Ladyman et al take this violation as reason to abandon the disjunctive approach, although they do note that on average, Kelvin’s law is not violated — the first scenario, in which the system does a huge amount of negative work, is just as likely to occur as the second scenario, and the negative work done in the first scenario far exceeds the positive work done in the second scenario.*

Having rejected the disjunctive approach, the authors then go on to argue for a formulation of the second law that covers probabilistic processes. Their revised law is as follows:

It is impossible to perform a cyclic process with no other result than that on average heat is absorbed from a reservoir, and work is performed.

(The original Kelvin law lacks the ‘on average’ caveat.) Having proposed and defended this revised law, they then use it to derive their main result, which is the expression for given in their answer 3 quoted above. As mentioned before, I have a problem with their derivation of their expression for , which I will explain in a later post. But even before we get to that derivation, notice that once they adopt the revised, probabilistic second law, the disjunctive approach, answer 1, is no longer problematic! As explained earlier, under the disjunctive approach, the second law may be violated half the time we perform the experiment, but on average it isn’t violated, since the other half of the time, a very large amount of net work is done on the system. So we no longer have a reason to think that the disjunctive approach, which seems to me the most immediately plausible, should be rejected. The modified law which Ladyman et al require for their derivation of the expression in answer 3 lets answer 1 off the hook.

*I think the rejection of the disjunctive approach is a bit of a fast move on the authors’ part, since I don’t think thermodynamics was set up to rule out cases where, due to a failure of statistics (in this case, a single-molecule gas makes it reasonably likely that all of the gas ends up on one side of the piston — note the extremely low probability of this happening in a normal gas), it is possible to occasionally violate the second law (though not in a pre-planned manner, i.e. there is no practically method to reliably repeat the violation).

References:

J Ladyman, S Presnell, A Short (2008). The use of the information-theoretic entropy in thermodynamics Studies In History and Philosophy of Science Part B: Studies In History and Philosophy of Modern Physics, 39 (2), 315-324 DOI: 10.1016/j.shpsb.2007.11.004

J Earman, J Norton (1998). Exorcist XIV: The Wrath of Maxwell’s Demon. Part I. From Maxwell to Szilard Studies In History and Philosophy of Science Part B: Studies In History and Philosophy of Modern Physics, 29 (4), 435-471 DOI: 10.1016/S1355-2198(98)00023-9

J Earman, J Norton (1999). Exorcist XIV: The Wrath of Maxwell’s Demon. Part II. From Szilard to Landauer and Beyond Studies In History and Philosophy of Science Part B: Studies In History and Philosophy of Modern Physics, 30 (1), 1-40 DOI: 10.1016/S1355-2198(98)00026-4