Examining a Probabilistic Doomsday Argument
Commentary

Is the sky falling? And if so, when? Even when they're baseless, constant reports about nuclear weapons proliferation, pandemic diseases and environmental catastrophes revive these perennial human questions and contribute to a feeling of unease.

So too did the recent passing of an asteroid almost 100 feet in diameter within 30,000 miles of the Earth. Such news stories make a recent abstract philosophical argument a bit more real.

Developed by a number of people including Oxford philosopher Nick Bostrom and Princeton physicist J. Richard Gott, the Doomsday Argument (at least one version of it) goes roughly like this.

On first consideration, it seems a straightforward question: How effective and safe is drug A in treating condition B? But the design and analysis of the clinical trials that set out to answer this question are far from straightforward, involving an overwhelming number of variables.

First, the subjects: Any group of human beings will show boundless variation in terms of both genetic makeup and nongenetic variation, such as age and lifestyle.

Then the disease: Behind the convenient categorization, each case of the "same" disease is as unique as the patient in terms of stage, underlying cause, previous treatment, and host interaction.

The impact of the tested drug will be influenced by dose, the patient's metabolism, genetics, and compliance with the trial regimen. Even seemingly trivial variability in the way individuals in different centers implement the trial design will add to the uncertainty and the inevitable errors in reading, recording, and analyzing data.

Extracting meaning from this noisy data is an industry in itself, and one that has its fair share of controversies. Most prominent are the definitions of significance, including the appropriateness of relying on P values; the interpretation of trial results involving multiple drugs; and the so-called meta-analysis of results from the same drug used in different trials.

When Justine Henin-Hardenne rips a cross-court forehand at the Australian Open or Tom Brady, the New England Patriots quarterback, dodges an onrushing defender, each looks like the very definition of living in the moment. Like other great athletes, they often appear to rely on speed, strength and lightning-fast reactions.

There seems to be little time for highly advanced quantitative analysis that weighs current observations against past experiences to suggest a plan of attack.

But this kind of analysis is precisely what the human brain does when facing a physical challenge, according to a study by two European scientists published in the current issue of Nature. The more uncertainty that people face — be it caused by wind on a tennis court, snow on a football field or darkness on a country highway — the more they make decisions based on their subconscious memory and the less they depend on what they see.

Among researchers, the combining of new information with conventional wisdom is known as Bayesian analysis, and it has become increasingly popular in recent years. Once controversial, because it muddies supposedly pure scientific data with subjective opinion about which prior research is relevant to a particular study, it has gained adherents as the explosion of computing power has allowed the method's complex formulas to be performed on a basic laptop computer.

If Daniel Gilbert is right, then you are wrong. That is to say, if Daniel Gilbert is right, then you are wrong to believe that a new car will make you as happy as you imagine. You are wrong to believe that a new kitchen will make you happy for as long as you imagine. You are wrong to think that you will be more unhappy with a big single setback (a broken wrist, a broken heart) than with a lesser chronic one (a trick knee, a tense marriage). You are wrong to assume that job failure will be crushing. You are wrong to expect that a death in the family will leave you bereft for year upon year, forever and ever. You are even wrong to reckon that a cheeseburger you order in a restaurant -- this week, next week, a year from now, it doesn't really matter when -- will definitely hit the spot. That's because when it comes to predicting exactly how you will feel in the future, you are most likely wrong.

Link to dissertation

If we want to have an educated citizenship in a modern technological society, we need to teach them three things: reading, writing, and statistical thinking.
- H.G. Wells

The authors present and test a new method of teaching Bayesian reasoning, something about which previous teaching studies reported little success. Based on G. Gigerenzer and U. Hoffrage's (1995) ecological framework, the authors wrote a computerized tutorial program to train people to construct frequency representations (representation training) rather than to insert probabilities into Bayes's rule (rule training). Bayesian computations are simpler to perform with natural frequencies than with probabilities, and there are evolutionary reasons for assuming that cognitive algorithms have been developed to deal with natural frequencies. In 2 studies, the authors compared representation training with rule training; the criteria were an immediate learning effect, transfer to new problems, and long-term temporal stability. Rule training was as good in transfer as representation training, but representation training had a higher immediate learning effect and greater temporal stability.

Fooled by Randomness: The Hidden Role of Chance in the Markets and in Life
Nassim Nicholas Taleb
Copyright 2001

The actual science of logic is conversant at present only with things either certain, impossible, or entirely doubtful, none of which (fortunately) we have to reason on. Therefore the true logic for this world is the calculus of Probabilities, which takes account of the magnitude of the probability which is, or ought to be, in a reasonable man's mind.
- James Clerk Maxwell (1850)