More on Frisch

More puzzlement about something in Frisch’s book. On p. 17:

These is disagreement between those who think that the domains of applicability of all scientific theories are essentially limited and those who have a hierarchical conception of science according to which the ultimate aim of science is to discover some universal theory of everything. My claim that classical electrodynamics is (and to the best of our knowledge will remain) one of the core theories of physics is independent of that debate… For, first, we need to distinguish between the content of a theory and our attitude toward that content. Even if we took the content of classical electrodynamics to be given by universal claims with unrestricted scope about classical electromagnetic worlds, our attitude can be that we accept or endorse those claims only insofar as they concern phenomena within the theory’s domain of validity.

Unlike Frisch, I do not see the point (other than the obvious rhetorical one of avoiding terminological disputes) of distinguishing between the content of a theory and our attitude towards it. Under Frisch’s account, that part of the content of the theory that encompasses claims about what the entire universe is like is irrelevant to our knowledge of the physical world, because we accept only claims of the theory that fall within the theory’s domain of validity. What, then, is the purpose of attributing these useless claims, which have no bearing on our knowledge of the world, to the content of the theory?

It may sound here like I am just arguing about terminology, but I do think this particular use of language, which is deeply entrenched in philosophy of physics, is pernicious. It is pernicious because by elevating some particular construal of the mathematical structure of the theory to be straightforwardly “the content” of the theory, one is led to an overemphasis on the mathematical structure as being informative about the world, while overlooking the informativeness inherent in how that structure fails to perfectly map onto the world. This terminology in effect provides a licence to focus solely on formal features of the mathematical structure as being the “essence” of the theory without seriously considering how the structure hooks on to the actual world, as opposed to how it hooks on to the philosopher’s possible worlds.

Frisch on reliable theories

There’s a really confusing passage in p. 42 of Mathias Frisch’s book on inconsistency in classical electrodynamics. He suggests, in response to the “problem” of inconsistency in classical electrodynamics, that we modify our account of theory acceptance:

this problem disappears if in accepting a theory, we are committed to something weaker than the truth of the theory’s empirical consequences. I want to suggest that in accepting a theory, our commitment is only that the theory allows us to construct successful models of the phenomena in its domain, where part of what it is for a model to be successful is that it represents the phenomenon at issue to whatever degree of accuracy is appropriate in the case at issue. That is, in accepting a theory we are committed to the claim that the theory is reliable, but we are not committed to its literal truth or even just of its empirical consequences. This does not mean that we have to be instrumentalists. Our commitment might also extend to the ontology or the ‘mechanisms’ postulated by the theory. Thus, a scientific realist might be committed to the reality of electrons and of the electromagnetic field, yet demand only that electromagnetic models represent the behavior of these ‘unobservables’ reliably, while an empiricist could be content with the fact that the models are reliable as far as the theory’s observable consequences are concerned.

If acceptance involves only a commitment to the reliability of a theory, then accepting an inconsistent theory can be compatible with our standards of rationality, as long as inconsistent consequences of the theory agree approximately and to the appropriate degree of accuracy… our commitment can extend to mutually inconsistent subsets of a theory as long as predictions based on mutually inconsistent subsets agree approximately.*

What confuses me about this is that I do not know what Frisch could mean by a theory being reliable apart from its consistently producing predictions that agree with experiment. Frisch wants to avoid instrumentalism by claiming that in accepting a theory, all we are committed not just to the observable consequences of the theory, but also possibly to the reality of the ontology and mechanisms of the theory. That is, in accepting the theory of electrodynamics, we might also be committed to the claim that electromagnetic models represent the behavior of ‘unobservables’ like ontology and mechanisms reliably. But what does it mean to represent reliably, apart from being a representation that reliably leads to predictions that agree with experiment? What does Frisch mean in the excerpt above by “represents the phenomenon at issue to whatever degree of accuracy is appropriate”? How can degrees of accuracy be attributed to representations over and above the accuracy of their experimental predictions?

Incidentally, I’m appalled at how expensive Frisch’s book is now. I bought it for $9 on Amazon when OUP slashed prices after having decided to stop printing it. Now it costs $60. The Kindle Edition costs $53.72!

* Frisch, M. (2005). Inconsistency, Asymmetry, and Non-Locality: A Philosophical Investigation of ClassicalElectrodynamics. Oxford University Press, USA.

What is a ‘fundamental theory’?

‘Fundamental theory’ is a phrase that is often used by philosophers and scientists alike, but I’m not sure what they mean by that. In this paper, Mathias Frisch writes:

In the case of non-fundamental theories, domain restrictions may also include restrictions to certain length- or energy-scales.

This suggests that fundamental theories are supposed to apply to all length- and energy-scales.

In the paper that Frisch is replying to, Gordon Belot writes:

We will of course need further principles that demarcate the domain of applicability of each theory — but every non-fundamental theory involves such principles.

Do fundamental theories, then, have no such restrictions on their domains of applicability? If so, does that mean that fundamental theories are just ‘theories of everything’? Does it also mean that we have no real examples of fundamental theories for our world, since all of our theories apply only to limited kinds or arrangements of matter?

Elena Castellani gives a quite different account of ‘fundamental physics’:

“fundamental physics” is the physics concerned with the search for the ultimate constituents of the universe and the laws governing their behavior and interactions. Fundamentality, on this view, is the prerogative of the physics of smaller and smaller distances (or higher and higher energies), and, accordingly, particle physics and cosmology are currently identified as the fields where the quest for the “final theory” takes place.

I find the quick move from ‘fundamentality’ to ‘finality’ in her last sentence quite intriguing. But that aside, note that Castellani here takes fundamental physics to refer to the study of the “ultimate constituents” of the universe and their behaviour. This characterisation is quite different from Frisch’s — Castellani’s ‘fundamental physics’ may involve a restriction to certain length scales (the very small). But perhaps she has a notion of ‘fundamental theory’ that is distinct from that of ‘fundamental physics’.

In fact, although they never explicitly state this, the discussion between Frisch and Belot regarding the consistency of classical electrodynamics often refers to a certain subset of the equations of classical electrodynamics as characterising the ‘fundamental theory’, with other subsets being non-fundamental theories of classical electrodynamics. But we know that classical electrodynamics has a limited domain of applicability. So I’m not sure what they mean by ‘fundamental theory’ in this context.

There is also a notion of ‘fundamental’ at work that talks about theories being more or less fundamental than other theories, but may not call any one theory ‘fundamental’ tout court. Stephan Hartmann, for example, characterises one theory as ‘more fundamental’ than another if it covers a broader range of energy scale which includes the energy scales at which the ‘less fundamental’ theory is valid. In the same paper, Hartmann quotes Steven Weinberg as claiming that a more fundamental theory is ‘on a level closer to the source of the arrows of explanation than other areas of physics’. However, this characterisation doesn’t mesh so well with debates in philosophy of science about whether a less fundamental theory can explain phenomena better than a more fundamental theory. If theories are to be defined as more fundamental according to the ‘arrows of explanation’, then it is simply by definition that there are no phenomena that can be explained better by less fundamental theories.

Next, we have Carlo Rovelli, who suggests that a limited domain of validity renders a theory non-fundamental, yet claims that QFT is “the fundamental theory of motion” (p. 257 of the linked book):

General relativity cannot be seen as a ‘fundamental’ theory since it neglects the quantum behavior of the gravitational field, but many of the directions that are explored with the aim of finding a quantum theory of the gravitational field and/or extending the Standard Model — perhaps to a theory of everything — are grounded in QFT. Thus, one may regard QFT as the fundamental theory of motion at our present stage of knowledge — playing the role that Newtonian mechanics and its Lagrangian and Hamiltonian extensions played in the eighteenth and nineteenth centuries.

Rovelli seems to be implying here that a ‘fundamental’ theory is one that proves to be fruitful in a large number of ways — at least that’s what I take his comparison with Newtonian mechanics, and his quip about the current role of quantum field theory, to imply. Yet he disqualifies GR as a fundamental theory on account of its neglect of quantum behavior.

So, what is a fundamental theory?

Metaphysics and Effective Theories

I’ve lately been finding it really difficult to get myself interested in alleged metaphysical issues stemming from assuming that a certain physical theory applies to the entire universe. Many of the ‘philosophical’ problems of quantum mechanics, for example, are of this ilk, as are many in algebraic quantum field theory and general relativity.

The standard practice in philosophy of physics is to use a so-called fundamental theory for your metaphysical inferences. Effective theories are for the most part ignored. The intuition is that the ‘fundamental theory’ describes ‘what is really out there’ while effective theories are somehow more phenomenological, or derivative of the fundamental theory (and hence need not be considered in addition to the fundamental theory).

A question arose in class today as to why is it that people who worry about entanglement in quantum mechanics typically worry about it in relation to special relativity rather than general relativity. The standard answer is that gravity is an effect that is negligible in the entanglement experiments we are considering, so we do not have to worry about what a quantum theory of gravity would have to say about the issue.

That got me wondering about how far someone could use that answer and still maintain that it is useful to figure out what the metaphysics of our world is by supposing that quantum mechanics applies to the entire universe. My worry is this. By using the ‘gravity is negligible’ reason, one is admitting that quantum mechanics is really just another effective theory — it has a limited domain of application. If so, then either

1. One thinks that in general it is legitimate to derive metaphysical conclusions using effective theories, or
2. One thinks that there is something special about quantum mechanics as an effective theory, which allows one to derive metaphysical conclusions from it, as opposed to other effective theories that are typically ignored (e.g. effective field theories).

If one goes with 1., then it seems to me that the right way to do scientifically-informed metaphysics is to take the various theories we have as each being informative about their respective domains of applicability. This has the implication that we should not be applying quantum mechanics to the entire universe and taking the metaphysical implications of that seriously. For it is classical theories that are most effective at large size scales, not quantum mechanics.

As for 2., I am still struggling to imagine what could be special about quantum mechanics that licenses us to treat it in a different way from other effective theories. One possible reason is that one thinks that the most important aspects of quantum mechanics will still persist in a ‘final theory’ which applies to the entire universe. But whatever these preserved aspects are, it’s not clear to me that they are the same aspects as those that lead to the traditional philosophical problems in quantum mechanics. It might be that the mathematics of the final theory is such that the problems with locality and whatnot that manifest themselves in quantum mechanics are somehow dodged. One can apply the same consideration to other issues in philosophy of physics. Maybe underdetermination in general relativity won’t actually translate to an underdetermination problem in the final theory.

In addition, if you look at the history of physics, it doesn’t seem to me that the aspects of older theories that are preserved in newer theories are those which tend to preserve philosophical problems in the older theories. It doesn’t seem to me as though any of the aspects of classical physics that are preserved in quantum mechanics are those that are philosophically problematic for either theory. In other words, the robust aspects of physical theories often aren’t those that lead to traditional philosophical problems.

Someone help me out here. I seem to be missing out on a lot of fun by being so pessimistic about this enterprise of reading metaphysics off ‘fundamental’ theories.

Formalist Analogies in Statistical Mechanics

I recently read Mark Steiner’s neat little book on the applicability of mathematics in physics. His main thesis is that the ways in which mathematics is successfully applied in physics are often anthropocentric. He takes this as a strike against naturalism.

One example of anthropocentric reasoning he identifies is the use of what he calls formalist analogies in the discovery/construction of new theories. One example that bugs many philosophers is “quantization”, where the quantum mechanical description of a system is derived by considering its classical description and replacing classical observables with quantum mechanical operators. This technique was introduced by Heisenberg, whose heuristics I expressed amazement at in my last post. There are problems with this heuristic, such as how to deal with descriptions containing products of classical variables, given that quantum mechanical operators don’t necessarily commute, but there are some standard workarounds that seem to work for most cases. Those problems are not really the issue Steiner is getting at, though. His issue is that the matrices Heisenberg uses to replace classical observables “have no independent physical meaning”; they are mere formalisms. The matrix equation one gets from a quantization of a classical equation is parasitic on the classical equation, which is itself “false” according to quantum mechanics. This lack of independent physical meaning, Steiner argues, means that we are not entitled to use induction to infer that since quantization works in certain model cases, it will work for all cases.

I’ve always been disturbed by the approach to statistical mechanics that makes use of Gibbsian ensembles, particularly the grand canonical ensemble. Part of my discomfort with it may be because many textbooks introduce them using formalist analogies, in Steiner’s sense. Gibbs himself was thoroughly instrumentalist about the ensembles and did not ascribe any physical meaning to them, but modern textbooks are liable to be more cavalier about physical meaning. Thus, one often finds them treating the ensembles as more than just a calculational technique, which I think treads into formalist analogy territory.

Gibbs emphasizes throughout his classic monograph that his ensembles are purely imaginary and meant to make calculations easier. However, later textbooks have a tendency to try to justify the use of ensembles by a mixture of physical and formal analogies. For example, textbook authors often speak of the equilibrium ensemble derived from combining two grand canonical ensembles (Tolman is one example). They take the resultant ensemble as the equilibrium that would result from combining a representative from one ensemble with that from another. Taken merely as a calculational tool, this is unproblematic. The formalistic reasoning comes into play when the outcome of the interaction of ensembles is straightforwardly taken, without further justification, to represent the outcome of the interactions of actual systems. For, as in the quantization analogy, the ensemble is parasitic on the actual system for its physical relevance. When two systems interact, we do not have two ensembles interacting. So there is no physical case to be made that the outcome of the ensembles’ interaction also represents that of the systems’ interaction. Just as the success of quantization in a few cases doesn’t seem to give us reason to expect a successful induction to all cases, the success of representing a system on its own with an ensemble doesn’t seem to give us reason to expect a successful induction to cases where multiple systems interact.

Certainly there exist authors who are more careful about using ensembles. Fowler for example justifies their use not by claiming that the entire ensemble represents the system of interest, but rather that the system is itself a small part of some larger system that has the characteristics of an ensemble. This makes more physical sense, but it means checking for more physically realistic conditions that your system of interest must fulfill, before applying ensemble methods. In Fowler’s case, he requires that the system of interest is one of many subsystems of a large ensemble-like system, where the subsystems exchange only small amounts of energy — small compared to the total energy of the ensemble. At the same time, however, their interactions with one another must be significant enough to allow the entire ensemble to attain an equilibrium state.

Newer textbooks, however, have a tendency to simply introduce Gibbsian ensembles, without checking for physical sense and the restrictions that must accompany them, and “justifying” them with mere formal analogies. One wonders what the point of such “justifications” is — I prefer Gibbs’ honest admission that he introduces ensembles only because they give him the correct answers.

Even more annoying are cases where a “justification” for using an ensemble is introduced with reference to a realistic physical model, but the ensemble is then used for examples where the physical conditions in that model, the conditions that were relevant to the justification, do not hold! For example, Pathria introduces the grand canonical ensemble by considering a system exchanging particles and heat with a large reservoir. However, all the problems he next considers, to which the grand canonical ensemble is applied, are cases where particle number is conserved! The only exception is an example of adsorption of particles on a surface, which appears as an exercise at the end of the chapter. We know that the grand canonical ensemble gives us the right answers even for systems that have a constant number of particles because for many-particle systems, the equilibrium ensemble contains is composed mostly of systems with the “equilibrium” number of particles, so when you average over the ensemble to get the equilibrium number, systems with a non-equilibrium number of particles contribute nearly nothing to the average. But this merely justifies the grand canonical ensemble as a calculational trick and is wholly separate from the physical model that was used to justify the ensemble method, a model whose salient features were then thoroughly ignored when the ensemble method was applied to other systems.

It feels like cheating.

I continue to be amazed at the flimsiness of the heuristics that physicists use, often successfully, to make important theoretical progress. A particularly shocking example I’ve just read is Heisenberg’s “discovery” that systems with symmetric wavefunctions correspond to those that obey Bose-Einstein statistics, and that those with anti-symmetric wavefunctions correspond to those that obey Pauli’s exclusion principle. He does not refer to Fermi-Dirac statistics since this was before Dirac “discovered” them, and Fermi’s discovery was also published in German only later (although he had published it in Italian earlier).

Why it feels like cheating:

1. The entire paper is based on the analysis of systems of coupled harmonic oscillators. He gives a quantum mechanical treatment of them that results in two groups of possible solutions, and shows that transitions between solutions can take place only between the members of each group. He then notes that we see only one of the two possible systems of ortho- and para-helium in nature, and suggests that this one-sidedness is due to the dichotomy he’d derived from his model. He then proceeds to generalise the dichotomy to all systems in nature:
   

   For the helium spectrum it is an empirical fact that only one system exists… that the other systems are not realised in nature. In fact it seems to me to indicate — if we assume, that the results we have derived for two systems can be generalised to arbitrarily many systems — on the one hand, an actual connection between the highlighted quantum mechanical indeterminacy [between which type of system exists], and on the other hand, the Pauli rule and the Einstein-Bose counting.

   (Pardon my amateur translation.) No justification for the generalisation exists in the paper.

2. Shortly after he admits:
   

   Grounds that this is the only system, of all the possible quantum mechanical solutions, that occurs, will scarcely be derived from the simple quantum mechanical calculation.

3. Despite all that, he still feels justified to extend the conclusions of his model into the realm of metaphysics:
   

   [The symmetric/anti-symmetric restrictions on wavefunctions] mean that it makes no physical sense to speak of the movement or the matrix representing the movement of an individual electron or of the matrix of any non-symmetric function of electrons in a system of atoms… Therefore e.g. the exchange relations in their familiar form also generally contain no physical sense

Of course, there is no question about the subsequent empirical success of relating anti-symmetric wavefunctions to Fermi-Dirac statistics and symmetric wavefunctions to Bose-Einstein statistics, but it remains amazing to me that such heuristics and casual generalizations as the ones Heisenberg uses are so successful. I do not think this is a one-off occurrence, either. In fact it seems to me that most theoretical development in physics proceeds this way, particularly the “revolutionary” developments.

Dirac, P. (1926). On the Theory of Quantum Mechanics Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character (1905-1934), 112 (762), 661-677 DOI: 10.1098/rspa.1926.0133
Fermi, E. (1926). Zur Quantelung des idealen einatomigen Gases Zeitschrift für Physik, 36 (11-12), 902-912 DOI: 10.1007/BF01400221
Heisenberg, W. (1926). Mehrkörperproblem und Resonanz in der Quantenmechanik Zeitschrift für Physik, 38 (6-7), 411-426 DOI: 10.1007/BF01397160

Landau on the second law

I was browsing through a collection of Landau’s papers and came across one of the more bizarre explanations of the second law of thermodynamics that I’ve seen. The few people who frequent this blog probably don’t need this explained to them, but the reason why many philosophers think that the second law needs ‘explaining’ is that the fundamental laws of classical and quantum mechanics are time symmetric, but the second law is not. If we think that all physical phenomena are completely described by the fundamental laws, then it seems puzzling that one can observe large scale time asymmetry when said laws are time symmetric.

Another way of stating that puzzle is that if one is given an isolated system in a state of less-than-maximal entropy, then standard applications of statistical mechanics lead to the conclusion that the system is likely to be found in a state of higher entropy in both directions in time — the past and the future. Landau accepts this as well. He then takes the problem of explaining observed monotonic change in time (a popular interpretation of the second law) to be one of explaining why all isolated subsystems seem to undergo monotonic change in the same direction. He offers the following argument:

If a statistical body is isolated from its surroundings for a period which is not too large compared with the relaxation time,* its entropy changes monotonically. If two such bodies are isolated, then the monotonic changes cannot occur in mutually opposed directions. This is immediately clear, since these two systems can be connected by an arbitrarily weak coupling into one single system, at any rate when their periods of isolation are simultaneous or partly so. If we apply the proof introduced above of the monotonic entropy change in this case of two weakly linked parts of one system, then we can see immediately that the directions of the entropy changes of the parts cannot be opposed to each other. If different systems are not simultaneously isolated from their surroundings, then we can introduce to the argument any necessary intermediary steps. We can now see that either the entropy of every statistical body which is not isolated from the universe for too long is constant, or there is a particular universal time direction for entropy increase.

This is weird since the clause about ‘relaxation time’ (see footnote) is a provision for the [supposed] fact that on time scales larger than the relaxation time, there isn’t a monotonic increase in entropy. So the ‘universal time direction’ would in this argument hold only for the system’s relaxation time. Yet, it’s also supposed to hold for all isolated systems that could be potentially coupled to the system in consideration! So, all such potentially coupled systems will undergo monotonic increases in entropy during the same time scale as our model system’s relaxation time!

It gets more bizarre. Recall that in the passage above, Landau concludes that either

1. ‘the entropy of every statistical body which is not isolated from the universe for too long is constant’, or
2. ‘there is a particular universal time direction for entropy increase.’

Landau opts for 2. Here’s how:

Everyday experience shows, however, that [1.] is not the case. This alone is quite sufficient (even without Bohr’s hypothesis that the inner layers of the stars may only be described in terms of relativistic quantum theory) to conclude that there are at least regions in the world which do not obey the statistics.** This makes the two directions of the time axis non-equivalent; the direction which is associated with the increase of the entropy of a system isolated over not too long a period form its surroundings can be defined as the future. For classical mechanics the conceptions of past and future are entirely without meaning. Only then do we obtain the second law of thermodynamics as the law of increase of entropy. This laws, as well as the existence of past and future observed in everyday experience, is only possible because the world as a whole does not obey the laws of thermodynamics.

This is the first time I’ve seen a claim like this: that entropy increases monotonically with time only because the laws of thermodynamics don’t apply to the entire universe! I must admit that because of the problems mentioned above, I can’t take any of Landau’s suggestions seriously, but the paper was a highly amusing read, and very reflective of how physicists tend to be much more imaginative than philosophers of physics.

*This clause about ‘relaxation time’ is supposed to get around the notion that the evolution of a system’s entropy with time should appear time-symmetric on the large scale, as a series of mostly small fluctuations away from equilibrium and back again. If we look at the system on a time scale close to the relaxation time, then we should be ‘within’ that portion of a fluctuation that involves monotonic increase or decrease of entropy with time.

**Landau states earlier in the paper that ‘in relativistic conditions… thermodynamic considerations [are] no longer valid’, and that he does not ‘expect thermodynamics to be upheld in the case of relativistic quantum theory’.

M. Bronstein, & L. Landau (1933). Über den zweiten Wärmesatz und die Zusammenhangsverhältnisse der Welt im Großen Physikalische zeitschrift der Sowjetunion, 4

Discrete observations and classical confidence intervals

In particle physics, experimentalists often aim to set limits on certain physical quantities, in part to verify theories. Say a theory predicts that a particle called Gobbledygook has a 10^-8 chance of decaying into two Gooks and a chance of decaying into three Gobbles. Often, the ratio between these two decay modes are closely related to important parameters in the theory. Experiments that try to set limits on the ratios of these decays can therefore give us an idea of the range of values in which those parameters fall. The fraction of total decays that a particular decay mode takes up is called the branching ratio of that decay mode.

These experiments proceed by creating a huge number of Gobbledygook decays, and counting the number of these decays that (say) result in two Gooks. The eventual count is therefore a discrete quantity — one cannot count a fractional number of decays. The branching ratio itself, which is what the experimenters try to set a limit on, is not a discrete quantity. So the limits that experimenters put on branching ratios are not subject to the restriction of discreteness — they can take on a range of continuous values.

In classical statistics, confidence intervals have the following significance. A 90% confidence interval means that if I carry out a large number of experiments and set a 90% confidence interval in each experiment about the quantity I’m measuring, then 90% of those confidence intervals will contain the actual value of the quantity I’m measuring. That is, classical confidence intervals say something about the expected coverage of the actual value that is generated by a particular method of constructing confidence interval.

So let’s say I want to put an upper limit on the branching ratio of a particular decay mode. I measure the number of such decay modes in my sample of decays, , and find that . I know that the decay mode is a Poisson process with unknown true mean , i.e. . To set a 90% confidence level upper limit on , I put and solve for . This gives me the upper limit .

Up to this point, we haven’t considered uncertainties due to the experimental setup. If there are no uncertainties whatsoever, that is, if the experimental apparatus and data analysis are of infinite precision, then the above method of constructing a 90% confidence interval, if repeated, will in fact lead to 90% of confidence intervals constructed this way covering .

However, no experiments have infinite precision, so we have to take uncertainties into account. But the classical 90% confidence interval we get when we take experimental uncertainties into account in fact leads (in the above example) to u₂ < 2.3, a tighter limit than the limit that an experiment with infinite precision would lead us to set! This, as Robert Cousins writes, is unacceptable since

if two experiments each find and have the same , the poorly calibrated one will report a more restrictive limit than the superbly calibrated one.

That is, we’d expect that the “more precise” experiment would allow us to place a stricter limit on the branching ratio, yet it turns out that with classical confidence intervals, the less precise experiment gives us a stricter limit!

Here’s how that happens. For the infinitely precise experiment, the 90% confidence interval is as described above. We want to measure the branching ratio , where is the true sensitivity of the experiment. In the infinitely precise experiment, there is no uncertainty in . Thus 90% of confidence intervals about the measured branching ratio will cover . 10% will not.

Now suppose we don’t know the true sensitivity . We can only estimate it by . Suppose . Suppose further that or , that is, is close to 2.3 relative to . Then the percentage of experiments that will observe is very close to 90%. When we construct the confidence intervals about from these experiments, their upper limit will be or greater, so nearly all of the 90% will cover . In the remaining 10% of experiments where , about half of the confidence intervals will cover — due to the term in the sensitivity. Thus the total coverage of will be approximately (90+5)%=95% — not 90%! A 90% confidence interval for the experiment with uncertainty , according to Cousins, would result in an upper limit of , stricter than the that one gets in the infinitely precise experiment!

Cousins says that this strange result is due to the discrete nature of observations in a Poisson process. I think of it intuitively this way. The discreteness of the observations means that with , about 10% of experiments will throw up the result . Because of the symmetric uncertainty about , about half of these will cover . Now, if were a continuous variable (excuse this rather dubious counterfactual), many of these incidences of would instead be spread over a range of positive values of . These incidences would have limits higher than the for , so fewer of them would cover compared to the discrete case. Thus, the discrete nature of the observations leads to over-coverage.

Note the occurrence of overcoverage does not depend on being close to 2.3. But the effect is magnified the closer is to 2.3.

Cousins uses this anomaly — that a “more precise” experiment can actually lead to less stringent limits on branching ratios — to argue that particle physicists should employ Bayesian statistics instead. But Bayesian statistics comes with its own collection of problems, the most obvious one being the need to choose a prior. This can sometimes be an “advantage”. In experimental particle physics, the Particle Data Group is a particularly important organisation. Every year, it publishes a Review of Particle Physics that is the “bible” for experimental particle physicists — among other things, it contains all the “accepted” values of physical constants and parameters relevant to particle physics. When Cousins wrote his paper, the PDG’s weighted average over experiments for the squared mass of the neutrino, with a central 68% classical confidence interval, was . That is, the entire confidence interval was in an “unphysical” region! If one uses a prior that is zero for values of , then one can rule out such "unphysical" confidence intervals. But this still leaves the question of whether the prior for the "physical" region should be uniform in , , or something else. Cousins reports that "the consensus view settled on , but the fact that the upper limit depends on this choice remains unsettling to many".

What I find most interesting about this statistical curiosity is the tensions at work in the desiderata for published limits on quantities like branching ratios. On the one hand, it would be nice to have a pithy description that is uniform for all the branching ratios listed in the Review of Particle Physics — all with a weighted average and the appropriate uncertainty associated with a standardised confidence level. That would be great utility for those looking for a quick overview of the experimental situation, say in order to jot down some rough pen-and-paper estimates in a related calculation. On the other hand, these pithy descriptions leave out the intricacies described in Cousins’ paper, imparting a perhaps misleading objectivity to the reported values. Recall that Cousins balks at accepting a method that leads to an experiment with infinite precision being less stringent with its limits than one with finite precision. I suspect that’s because he’s acknowledging the experiment as imparting authority to its reported mean value and confidence interval in its own right, not as just another statistic in the hypothetical ensemble of experiments that together satisfy the requirements of classical confidence intervals. If one takes the ensemble point of view seriously, then it’s not clear that Cousin’s worry matters. Of course, there is a whole other question about whether we should really be thinking in terms of large ensembles of experiments in experimental particle physics, given that the difficulty and expense of such experiments ensure that we do not have such large ensembles in practice.

Cousins, R. (1995). Why isn’t every physicist a Bayesian? American Journal of Physics, 63 (5) DOI: 10.1119/1.17901

Sean Carroll’s From Eternity to Here

I recently finished reading Sean Carroll’s new book, From Eternity to Here. Peter Woit opined that the book was “an extended essay in the philosophy of science” and was curious to know what philosophers would think of it, so I thought I’d give it a shot. While the questions that Carroll addresses are questions that are discussed by philosophers of physics, the book lacks good arguments for the conclusions that Carroll draws. He makes many sweeping claims, many of which have been disputed by philosophers, but he makes no engagement with those disputes. The arguments he provides are not new in the literature and are much better defended elsewhere. So it’s certainly not a new contribution to the philosophy of physics literature, and if it is philosophy of science, is philosophy of science badly done.

First, a short sketch of Carroll’s argument. Carroll essentially takes David Albert’s line that the tension between the time-asymmetric second law of thermodynamics and the time-symmetric laws of classical mechanics can be resolved by postulating low-entropy initial conditions for the universe — a postulate that has come to be called the Past Hypothesis in the philosophical literature. On top of that, Carroll argues that those initial conditions themselves need explanation, and the explanation he favours is a multiverse that buds off little “baby universes” that start off with low entropy initial conditions, We, Carroll suggests, are in one of those baby universes.

Carroll often makes a major philosophical point without sufficient justification and without addressing some obvious objections. Most but not all of the following instances where he does this are points are important for his main argument.

1. Explanation of “special” initial conditions of the universe. Craig Callender objected in his review of Carroll’s book to Carroll’s assumption that “special” initial conditions have to be explained at all. Callender himself argued this with more detail in a paper, and the issue of whether explanation is needed in this case is a matter of debate in philosophy of science. But Carroll doesn’t justify his assumption at all.
2. The Past Hypothesis. As mentioned, Carroll’s argument for the Past Hypothesis is basically David Albert’s. This argument has been criticized in various places. To his credit, in this case Carroll does mention the existence of those criticisms, in footnote #142, where he admits that the status of the Past Hypothesis is “not uncontroversial”. However, he does not engage with those criticisms.
3. The Past Hypothesis as a law of nature. Carroll states that there is a distinction in physics between laws and initial conditions, and that since the Past Hypothesis is an instance of the latter, it is not a law of nature. But this does not address the reason supporters of the Past Hypothesis think the Past Hypothesis is a law of nature. The reason David Albert, Barry Loewer and a few others think that the Past Hypothesis is a law of nature is because they hold a Humean, Mill-Ramsey-Lewis view of what the laws of nature are, and think that the Past Hypothesis would appear in the system of statements about the world that forms the best balance of simplicity and informativeness.* Carroll’s statement about laws versus initial conditions completely ignores the [prima facie, not unreasonable] view of laws of nature held by these people. Carroll’s objection to calling the Past Hypothesis a law of nature is premised on a different view of what the laws of nature are, a view that is neither articulated nor justified.**
4. The relation between information theoretic entropy and thermodynamic entropy. Carroll repeats Landauer and Bennett’s arguments for equating information theoretic entropy and thermodynamic entropy. These arguments have been criticised, and Carroll does not mention these criticisms. The equation of those two kinds of entropy is put forward as unproblematic when it is not.
5. The different arrows of time. Carroll frequently conflates the different arrows of time and makes big, unsubstantiated claims based on those conflations. A series of such claims is presented with hardly any support on pp. 40-41. Among the claims are:
   • The arrow of time*** “explains” why we remember the past but not the future.
   • The fact that our memories of the past are reliable is explained by the monotonic increase of entropy with time.
   • “We distinguish past from future through the relationship between cause and effect.” (Isn’t it the other way around?)
   • From the previous point, he thinks it follows that “part of the distinction we draw between ‘effects’ and ’causes’ is that ‘effects’ generally involve an increase in entropy”. (I don’t understand. How do causes not generally involve an increase in entropy? How are causes somehow exempt from the second law of thermodynamics? And what does he mean by “involve”?)
   • “Our notion of free will… is only possible because the past has a low entropy and the future has a high entropy.”

Overall, I think it’s a nice pop science book to read for laypeople who want to get their feet wet in some of the issues in the foundations/philosophy of statistical mechanics, but for those already familiar with the ins-and-outs of that field, and for philosophers in general, Carroll’s book is too sloppy to be of much help.

*See for example this paper by Loewer.

**It seems, among other things, to be a non-Humean view.

***It’s not clear at this point even what Carroll means by the arrow of time.

The purpose of physics graduate classes

I’m taking a graduate statistical mechanics course this semester. My first physics course in more than two years, and my first graduate physics course.

One reason I started disliking physics courses when I was an undergraduate was that class time was spent almost entirely on going through the details of the derivations in the textbook. If there’s anything guaranteed to send me to sleep, especially if it’s a 9.30am class, it’s someone at the board moving symbols here and there and reciting the arithmetical rules he’s using to move those symbols. Furthermore, the vast majority of derivations are mathematically straightforward and can be understood from a close reading of the textbook. I don’t need someone to go through what I can glean from reading the textbook on my own.

For whatever reason, I thought that graduate classes in physics would be better. Well, this one isn’t. The professor is going through nearly every single line in Pathria’s text. What’s more, he actually tells us to read the textbook beforehand because he doesn’t want us to be looking at the textbook figuring out the math while he’s “teaching”. But if I read the textbook beforehand (which I do), I understand the derivation, so I get bored when he comes to class and goes through the exact same derivation, except more slowly and in more painful detail. So far I’ve always ended up working on my problem sets during class instead, which I find a much more productive use of my time.

One point that Eriz Mazur makes in this excellent talk on science education is that in a humanities class, it’s standard to expect students to do the assigned reading. The class then proceeds with the assumption that students have done the reading. The instructor does not hold your hand and lead you through every line of the reading. It is also understood that if you don’t do the readings, it pointless to go to class because the class is going to assume you’ve at least grappled with them, and start off on that higher level.

The opposite happens in science classes. In the vast majority of physics classes I had, even if the professor recommends that you read certain parts of the textbook or lecture notes, s/he still leads you by the hand through the very material that you’ve just read. You could not read any of it and still get the contents of what you were supposed to have read through the lecture alone. (The one class I had that did not follow this mould, which was also my favourite class in any subject ever, was taught by someone who would receive negative feedback from most students about his more Socratic teaching style. People complained that he did not follow a textbook, but for me, it is exactly when someone follows a textbook that his lectures fail to add value.) The learning style encouraged by such teaching seems to be passive rather than active, compared to humanities classes.

All this may be excusable as a sop to undergraduates who are either too stupid or lazy to read textbooks on their own, but firstly, the same undergraduates are not treated as such by their humanities instructors, and secondly, why is this still going on in graduate classes? Why do graduate students have to be hand-held through a textbook? If you can’t read a textbook like Pathria on your own, should you even be in a physics PhD programme?

A useful contrast is with my philosophy graduate seminars. There, as with undergraduate humanities classes, you’re expected to do the readings before class. Discussions in class proceed on the assumption that you have done your readings. Students are typically asked to present some of the readings, and the presentations will have some sort of summary of the contents of the readings, but it’s nothing to the extent of the presenter going through step by step of the arguments in the readings, the way physics teachers go through the textbook derivations step by step. And of course, there is much discussion of the readings.

I discussed this with someone before, and the response was that in science, there is typically a definite “right answer” to questions, whereas in the humanities, most of the important answers are still unknown. The implication is that if there is a “right answer”, then this should be told to the students. In contrast, if the right answer is not known, then discussion might somehow bring one closer to it.

I suspect many people think of science education this way, and I think they’re deeply mistaken. The objective of science education as I see it is not to tell people the right answers. The objective is understanding. People may know the right answers without understanding why they are right. In physics, the analogue would be if a student could do all the problems set by her instructor by the simple expedient of applying certain formulaic rules, but does not understand why those rules hold. This was my situation for most of my physics classes, and was a huge contributor to why I became frustrated with physics. Of course, I did go to my professors outside of class to try to get a deeper understanding, but most of the time they could not answer my “conceptual” questions. They seemed to be prepared only to tell students how to apply certain rules, without being able to justify those rules themselves.

The peer discussions that Mazur and an increasing number of people who study science education are advocating go some way to helping students get to the answer on their own and, on the way, gaining a deeper understanding of why the answer is what it is. It makes science education more like humanities education.

Pushing the idea further, I see no reason why a graduate class in physics cannot be run like a graduate class in philosophy. Have the students read the relevant section of the textbook beforehand. This should be just as obligatory as doing the readings for humanities classes is. In class, ask if anyone has had any problems understanding the assigned readings. If someone has, try to straighten her out. Do not go through every step of the proofs in class, since it could (and should) well be the case that most students understand the proofs already. Class time should be used instead to discuss interesting implications of the proofs, setting the context for them, considering the assumptions they use and what implications those assumptions have, and so on. In fact, I also see no reason why the model of having students present on the assigned readings cannot be applied. Do physics professors have such low expectations of their students that they think they cannot learn on their own and have to be spoonfed just like undergraduates? Or is it I who has overly high expectations of physics graduate students?