Lindley's paradox

Lindley's paradox describes a counterintuitive situation in statistics in which the Bayesian and frequentist approaches to a hypothesis testing problem give opposite results for certain choices of the prior distribution. The problem of the disagreement between the two approaches was discussed in Harold Jeffreys' textbook [cite book
author=Jeffreys, Harold
title=Theory of Probability
publisher=Oxford University Press
date=1939] ; it became known as Lindley's paradox after Dennis Lindley called the disagreement a paradox in a 1957 paper [cite journal
author=Lindley, Dennis V.
title=A Statistical Paradox
journal=Biometrika
volume=44 |pages=187–192
date=1957] .

Description of the paradox

Consider a null hypothesis "H₀", the result of an experiment "x", and a prior distribution that favors "H₀" weakly. Lindley's paradox occurs when
# The result "x" is significant by a frequentist test, indicating sufficient evidence to reject "H₀", say, at the 5% level, and
# The posterior probability of "H₀" given "x" is high, say, 95%, indicating strong evidence that "H₀" is in fact true.

These results can happen at the same time when the prior distribution is the sum of a sharp peak at "H₀" with probability "p" and a broad distribution with the rest of the probability "1-p". It is a result of the prior having a sharp feature at "H₀" and no sharp features anywhere else.

Example: heavier-than-air flight

To see why this might happen, consider the example of an experiment in a very well-established field of science, such as gravitation, and consider the null hypothesis to be something like, "what we currently believe about gravity is true." The prior probability for this hypothesis is of course very large.

Now, suppose that in this particular experiment, something very unusual happens, and for whatever reason the results of observations appear to be inconsistent with gravitation as we understand it (e.g., a 1,000 ton metal structure lifts off the ground and flies at near the speed of sound). The conditional probability of this occurrence is small given the null hypothesis, and so a frequentist ought to reject the null hypothesis.

Yet a Bayesian knows that the prior probability of the null hypothesis is so high that this experiment is not sufficient to actually conclude that the null hypothesis is false, but rather that something unusual is going on (i.e. powered air flight). In practice, even scientists who consider themselves "frequentist" would adopt this methodology, considering information outside the experiment in drawing their inferences.

References

*