Information-theoretic entropy and thermodynamic entropy

There’s a long history of attempts to prove that information theoretic entropy is somehow connected to thermodynamic entropy. Most of these attempts have used Maxwell’s Demon or elaborate thought experiments involving single-molecule gases in boxes with removable walls and pistons and such. The likes of Charles Bennett have then gone on to spin out various implications of these thought experiments for reversible computing. I’m convinced by Earman and Norton’s arguments that these attempts are all misguided. Recently, Ladyman, Presnell and Short [preprint] made a new attempt to prove that information-theoretic entropy does apply to probabilistic mixtures of macrostates.

Ladyman et al consider a thermodynamic system C that is prepared in thermodynamic states or with equal probability. Suppose the states and have entropies and respectively. What entropy should we assign to C? Ladyman et al consider three possibilities:

1. We cannot assign a unique entropy to C since is either or but we do not know which. We can only proceed disjunctively by considering each possibility separately.
2. In our calculations we should set to equal the weighted average of and , namely .
3. In our calculations we should set to equal the weighted average of and plus an additional term that represents a contribution to the thermodynamic entropy due to the probability distribution itself. In this particular case, (where k is Boltzmann’s constant).

They argue that answer 1 would violate the Kelvin statement of the second law of thermodynamics. They put forward a revised version of the Kelvin law that would cover probabilistic processes of the sort considered above. They then use this revised law and a thought experiment to derive the expression for given in answer 3. The term in answer 3 is supposed to validate the connection between information-theoretic entropy and thermodynamics.

I have two problems with what they do:

1. Their revised version of the Kelvin law annihilates their objections to answer 1.
2. The thought experiment with which they derive the expression for could be altered so as to derive other expressions for that are not amenable to an information theoretic interpretation.

In this installment I’ll just elaborate on point 1. The second point will be explicated in a later post.

Ladyman et al’s dismissal of the disjunctive approach towards the entropy of C is built on a thought experiment using a single-molecule gas. Suppose one has an ideal gas consisting of a single molecule in a box, connected to a heat reservoir maintained at a constant temperature. The thought experiment proceeds as follows:

1. A partition is inserted reversibly into the middle of the box. The particle is now on one side of the partition or the other with equal probability.
2. A piston is pushed in isothermally from one side, say the left, up to a small distance from the partition.
3. The partition is removed and the gas is allowed to expand isothermally against the piston until the gas is returned to its original state.

Under the disjunctive approach, two things can happen. Either the particle is on the left side of the partition, so that the work done in pushing the piston in is very large (tending to infinity as the distance the piston stops from the partition gets smaller), or the particle is on the right side of the partition, so that no work is done in pushing the piston in, but the gas performs some work in pushing the piston back out. In the second scenario, a cyclic process occurs in which a net positive amount of heat is absorbed from the reservoir and the system does a net amount of work. This violates Lord Kelvin’s formulation of the second law of thermodynamics. Ladyman et al take this violation as reason to abandon the disjunctive approach, although they do note that on average, Kelvin’s law is not violated — the first scenario, in which the system does a huge amount of negative work, is just as likely to occur as the second scenario, and the negative work done in the first scenario far exceeds the positive work done in the second scenario.*

Having rejected the disjunctive approach, the authors then go on to argue for a formulation of the second law that covers probabilistic processes. Their revised law is as follows:

It is impossible to perform a cyclic process with no other result than that on average heat is absorbed from a reservoir, and work is performed.

(The original Kelvin law lacks the ‘on average’ caveat.) Having proposed and defended this revised law, they then use it to derive their main result, which is the expression for given in their answer 3 quoted above. As mentioned before, I have a problem with their derivation of their expression for , which I will explain in a later post. But even before we get to that derivation, notice that once they adopt the revised, probabilistic second law, the disjunctive approach, answer 1, is no longer problematic! As explained earlier, under the disjunctive approach, the second law may be violated half the time we perform the experiment, but on average it isn’t violated, since the other half of the time, a very large amount of net work is done on the system. So we no longer have a reason to think that the disjunctive approach, which seems to me the most immediately plausible, should be rejected. The modified law which Ladyman et al require for their derivation of the expression in answer 3 lets answer 1 off the hook.

*I think the rejection of the disjunctive approach is a bit of a fast move on the authors’ part, since I don’t think thermodynamics was set up to rule out cases where, due to a failure of statistics (in this case, a single-molecule gas makes it reasonably likely that all of the gas ends up on one side of the piston — note the extremely low probability of this happening in a normal gas), it is possible to occasionally violate the second law (though not in a pre-planned manner, i.e. there is no practically method to reliably repeat the violation).

References:

J Ladyman, S Presnell, A Short (2008). The use of the information-theoretic entropy in thermodynamics Studies In History and Philosophy of Science Part B: Studies In History and Philosophy of Modern Physics, 39 (2), 315-324 DOI: 10.1016/j.shpsb.2007.11.004

J Earman, J Norton (1998). Exorcist XIV: The Wrath of Maxwell’s Demon. Part I. From Maxwell to Szilard Studies In History and Philosophy of Science Part B: Studies In History and Philosophy of Modern Physics, 29 (4), 435-471 DOI: 10.1016/S1355-2198(98)00023-9

J Earman, J Norton (1999). Exorcist XIV: The Wrath of Maxwell’s Demon. Part II. From Szilard to Landauer and Beyond Studies In History and Philosophy of Science Part B: Studies In History and Philosophy of Modern Physics, 30 (1), 1-40 DOI: 10.1016/S1355-2198(98)00026-4

Emacs and Xfce

Per my last post, I spent the greater part of the morning customizing Xfce for my office desktop. An annoying quirk I noticed is that Xfce sets Alt-Del and Alt-Space as default keyboard shortcuts for some window manager functions. In Emacs, Alt-Del is supposed to delete the word before the cursor. I tried doing this several times today, only to find myself reduced to one workspace, with the programs from the other workspaces suddenly popping into the one I was working in. I eventually figured out that Alt-Del deletes a workspace from Xfce every time it’s invoked (it will not, of course, delete the only workspace left, if you have only one). Alt-Space, which sets markers in Emacs, does something more benign — I used it many times without noticing any changes.

In any case, to save my future self or others the trouble of Googling for the solution, you can restore the functionality of Alt-Del and Alt-Space in Emacs by customizing the keyboard shortcuts in Xfce’s Window Manager settings. Confusingly, Xfce has two places to configure keyboard shortcuts — one under the Keyboard settings, and one under Window Manager –> Keyboard. It’s the latter you need to disable the shortcuts that interfere with Emacs.

Another thing — the latest Xfce installations don’t seem to come with a default panel that includes shortcuts to the web browser and the terminal. You have to add them yourself to the empty panel, and to do this you need the command for the application. It took me some time to figure out what command would invoke a terminal, since I’ve never invoked a terminal from the command line! I’m using gnome-terminal now, but if you have it installed, you can use xfce-terminal, or even the minimalist xterm.

In other news, I am very, very sick of C++.

Enough Gnome

I’ve been having problems with Gnome and Firefox 3 crashing in Ubuntu Hardy Heron. The Firefox crashes don’t bother me much, since upon restart Firefox remembers the tabs that were open before the crash. Gnome crashing, however, is annoying — I lose all the processes that were running beforehand, and some programs cannot be restarted after I immediately log into a new session of Gnome (they can be restarted only after I restart the computer, rather than just starting a new Gnome session).

So I’m going back to Xfce. I’d used Xubuntu Dapper Drake before on the old desktop with limited RAM they dumped on me when I started work, and Xfce never crashed.

I also installed Xubuntu Intrepid Ibex on my 256MB RAM, ancient Dell laptop. It was practically frozen on Windows XP, but runs at a passable speed with Xubuntu. I was driven to do this because I’m sick of using Windows while Applecare takes forever with my Macbook.

I still prefer OS X to Ubuntu, but I’m not at all happy that Apple wanted to charge me the equivalent of US$200 to backup my data and reinstall the OS. (I told them to just change the defective hard disk, a free service under the warranty, and screw the data — I have all the important bits backed up anyway.) Also, I was inadvertently left with time to kill in an Apple store recently, and, strolling the aisles, was struck by how evil their marketing was. So I’m reconsidering giving more money to Apple for my next laptop.

Update: After 8(!) working days, I have my Macbook back with a new hard disk, and a Leopard installation as a bonus (I only had Tiger beforehand). Does not make up for the long delay and the generally bad service from the frontline staff, but a nice silver lining nonetheless.

Spotlight + torrent of PDFs = disaster

Spotlight goes crazy updating its indices when you’re downloading a good number of PDFs through Bittorrent. I just learned this the hard way. I left Bittorrent running, went off to run an errand, and when I returned, my hard disk was full thanks to Spotlight’s hyper-indexing. At length, the system hung, and I did a forced shut-down, only to get the dreaded question mark when I restarted. The good news is that Disk Utility tells me that my disk ‘appears to be OK’. With the install disc, I can even change my password for the startup disk. What I can’t seem to do, however, is to get it to boot.

I’m cheesed off not because of the probably lost data (all the files I need are backed up), but because of the time I’ll have to waste either reinstalling the system myself or going to the Apple shop and asking them if they can do anything better than a reinstallation.

Update: Attempting to start in safe mode indicated that it was a kernel panic, probably brought on by the full hard disk.

Britten’s Arrangements of Folk Songs

I’m listening to these for the first time in several months, and am struck anew by the genius in some of them. It’s all in the piano accompaniment. One of my favourites is that for The Ash Grove. The accompaniment is sweet and wistful but still in key for the first stanza, when the singer tells of his dear one. It does however start out already ‘out of sync’ with the singer’s melody. This contributes to the sense that the singer is speaking of a time and situation that is alien, that is not here and now. In the second stanza the melodies in the voice and piano are still out of sync, but now the harmony is off too. The singer retains the same harmony, but the accompaniment is dissonant. Thus even as the singer describes the still-beautiful scenery at the start of the stanza, the listener is beset with a sense of uneasiness — something is wrong, the piano is wrong, despite these nice words! When the singer tells of his sorrow, the harmonies become less alien but still not quite right — for there is no dissonance here with the pleasant scenery, but there is still the pain of sorrow, now in the open. For the final line, the harmonies come into line again — the maiden, we learn, is at rest.

Here is a recording (FLAC) of Peter Pears and Benjamin Britten performing this piece. And here are Pears and Britten performing the latter’s arrangement of O Waly, Waly. More of the unsettling harmonies again.

Learning General Relativity

Before I started on my post-graduation attempt to learn GR, I’d had only some grounding in the fundamentals of differential geometry. In my undergrad course where this was covered, we only got as far as vectors, covectors and (a bit of) tangent bundles. No physics involved, although there was a somewhat awkward attempt to do thermodynamics on manifolds.

I began this journey with a study group partner, whom I shall call A, whose main objective was to learn differential geometry rather than GR. At his recommendation, we started with Reyer Sjamaar’s freely available notes on manifolds and differential forms. I quite disliked this from the start; it introduces manifolds very late in and for the first few chapters you’re faffing about shifting the indices on differential forms! I could get no intuitive understanding at all from them, so after a while I gave up and suggested Schutz’s Geometrical Methods of Mathematical Physics. I actually quite liked this book. Schutz is pretty good at explaining things in intuitive terms. I like this kind of approach when I’m starting on a new subject, because I tend to learn better with an ‘examples first‘ approach than with an ‘axioms first’ approach. The book is clearly catered to physicists, though — mathematicians would find it unrigorous. A is a physicist himself, but he wasn’t happy with the lack of rigour. Three chapters in, A wanted to change books, in particular to Analysis, Manifolds and Physics. Now this book is clearly written by mathematicians, for mathematicians. In fact, some of the early parts introducing manifolds are hard to get through unless you’re already reasonably familiar with topology. The authors had an annoying habit of using theorems in topology to prove propositions in differential geometry. A was quite happy with the rigour in this when we started out, but we got lost so quickly (me having only the most cursory knowledge of topology and him having none) that he was only too happy to let me suggest the next route. While the book itself has a section summarising topology, it is a typical mathematician’s summary — exceedingly terse and, though possibly useful as a refresher to someone who has studied topology before, quite unhelpful to novices.

I had heard many good things about Wald’s General Relativity, the classic graduate text, so we tried that next. I think we both really liked it from the beginning, and perhaps only became frustrated in the last month or so. The first two chapters were great — good prose explanations, and the math is about the right level for those who have some experience with differential geometry. I definitely would not recommend it to a complete novice, though. At this point, we’re slogging through the problems at the end of Chapter 3. Chapter 3 itself was a long, hard slog for us — it’s where Wald introduces most of the apparatus needed to determine curvature, geodesics and such, and the proofs are involved. The problems, now, the problems. Along the way we acquired another study group participant, someone who’d done a GR course before but had forgotten most of it. He had little formal math background but had a great intuitive feel for just shifting indices around until they fell into an arrangement suitable for solving the problem. So he didn’t really understand the formal mathematical characteristics of manifolds and tangent vectors and such, and we had to clear up his confusions about those a few times, but he’s by far the most efficient at doing the problems. Indeed, you can do many of them by just shifting indices around in the right way. Even so, we’ve frustratingly spent about three hours per problem on average for this chapter, and the things we prove don’t even seem particularly physically or mathematically interesting! There’s a lot of counting of symmetries for very special, low-dimensional cases. Since we meet once a week for a few hours, we’ve basically spent the last month doing the first four problems at the end of Chapter 3. Depressing. We’ll press on, though, and hope we get better with practice.

At the same time, I am working my way alone through David Malament’s freely available notes (pdf) for his Foundations of GR course. I like them, perhaps because they closely follow the prose style and theoretical approach of the person who first taught me differential geometry. Lots of nicely worded intuition pumps, like Schutz, but more mathematically rigorous. The problems also seem easier and more interesting than those in Wald. (Since there’s no external evaluator to judge that I’ve mastered the subject, I’m not convinced that I understand it adequately until I can do all or at least most of the problems in whatever book or notes I choose to follow.) I’m only up to the chapter on Lie derivatives now, though, so for all I know they may turn tedious and unrewarding once we hit the Riemann tensor.

Late discovery

The Perimeter Institute for Theoretical Physics has an archive of audio, video, and presentation files of its seminars. It includes some plum conferences, such as Many Worlds at 50.

Not all its available recordings seem to be up there, though. I was reminded to look for more audio seminar files on their website only after I uncovered some old files from their Evolving Laws seminar in my iTunes library. The files there don’t seem to be on PIRSA.