de Finetti's theorem

de Finetti's theorem

In probability theory, de Finetti's theorem explains why exchangeable observations are conditionally independent given some latent variable to which an epistemic probability distribution would then be assigned. It is named in honor of Bruno de Finetti.

It states that an exchangeable sequence of Bernoulli random variables is a "mixture" of independent and identically distributed (i.i.d.) Bernoulli random variables – while the individual variables of the exchangeable sequence are not themselves i.i.d., only exchangeable, there is an underlying family of i.i.d. random variables.

Thus, while observations need not be i.i.d. for a sequence to be exchangeable, there are underlying, generally unobservable, quantities which are i.i.d. – exchangeable sequences are (not necessarily i.i.d.) mixtures of i.i.d. sequences.

Contents

Background

A Bayesian statistician often seeks the conditional probability distribution of a random quantity given the data. The concept of exchangeability was introduced by de Finetti. De Finetti's theorem explains a mathematical relationship between independence and exchangeability.

An infinite sequence

X_1, X_2, X_3, \dots \!

of random variables is said to be exchangeable if for any finite cardinal number n and any two finite sequences i1, ..., in and j1, ..., jn (with each of the is distinct, and each of the js distinct), the two sequences

X_{i_1},\dots,X_{i_n} \text{ and } X_{j_1},\dots,X_{j_n} \!

both have the same joint probability distribution.

If an identically distributed sequence is independent, then the sequence is exchangeable; however, the converse is false --- there exist exchangeable random variables that are statistically dependent, for example the Polya urn model.

Statement of the theorem

A random variable X has a Bernoulli distribution if Pr(X = 1) = p and Pr(X = 0) = 1 − p for some p ∈ (0, 1).

De Finetti's theorem states that the probability distribution of any infinite exchangeable sequence of Bernoulli random variables is a "mixture" of the probability distributions of independent and identically distributed sequences of Bernoulli random variables. "Mixture", in this sense, means a weighted average, but this need not mean a finite or countably infinite (i.e., discrete) weighted average: it can be an integral rather than a sum.

More precisely, suppose X1, X2, X3, ... is an infinite exchangeable sequence of Bernoulli-distributed random variables. Then there is some probability distribution m on the interval [0, 1] and some random variable Y such that

  • The probability distribution of Y is m, and
  • The conditional probability distribution of the whole sequence X1, X2, X3, ... given the value of Y is described by saying that
    • X1, X2, X3, ... are conditionally independent given Y, and
    • For any i ∈ {1, 2, 3, ...}, the conditional probability that Xi = 1, given the value of Y, is Y.

Another way of stating the theorem

Suppose X1, X2, X3, ... is an infinite exchangeable sequence of Bernoulli-distributed random variables. Then X1, X2, X3, ... are conditionally independent given the tail sigma-field.

Example

Here is a concrete example. Suppose p = 2/3 with probability 1/2 and p = 9/10 with probability 1/2. Suppose the conditional distribution of the sequence

X_1, X_2, X_3, \dots \!

given the event that p = 2/3, is described by saying that they are independent and identically distributed and X1 = 1 with probability 2/3 and X1 = 0 with probability 1 − (2/3). Further, the conditional distribution of the same sequence given the event that p = 9/10, is described by saying that they are independent and identically distributed and X1 = 1 with probability 9/10 and X1 = 0 with probability 1 − (9/10). The independence asserted here is conditional independence, i.e., the Bernoulli random variables in the sequence are conditionally independent given the event that p = 2/3, and are conditionally independent given the event that p = 9/10. But they are not unconditionally independent; they are positively correlated. In view of the strong law of large numbers, we can say that

\lim_{n\rightarrow\infty} \frac{X_1+\cdots+X_n}{n} = \begin{cases}
2/3 & \text{with probability }1/2, \\
9/10 & \text{with probability }1/2.
\end{cases}

Rather than concentrating probability 1/2 at each of two points between 0 and 1, the "mixing distribution" can be any probability distribution supported on the interval from 0 to 1; which one it is depends on the joint distribution of the infinite sequence of Bernoulli random variables.

The conclusion of the first version of the theorem above makes sense if the sequence of exchangeable Bernoulli random variables is finite, but the theorem is not generally true in that case. It is true if the sequence can be extended to an exchangeable sequence that is infinitely long. The simplest example of an exchangeable sequence of Bernoulli random variables that cannot be so extended is the one in which X1 = 1 − X2 and X1 is either 0 or 1, each with probability 1/2. This sequence is exchangeable, but cannot be extended to an exchangeable sequence of length 3, let alone an infinitely long one.

Extensions

Versions of de Finetti's theorem for finitely exchangeable sequences[1] and for Markov exchangeable sequences[2] have been proved by Diaconis and Freedman.[3]

Two notions of partial exchangeability of arrays, known as separate and joint exchangeability lead to extensions of de Finetti's theorem for arrays by Aldous and Hoover.[4]

The computable de Finetti theorem shows that if an exchangeable sequence of real random variables is given by a computer program, then a program which samples from the mixing measure can be automatically recovered.[5]

In the setting of free probability, there is a noncommutative extension of de Finetti's theorem which characterizes noncommutative sequences invariant under quantum permutations.[6]

References

  1. ^ Diaconis, P.; Freedman, D. (1980). "Finite exchangeable sequences". Annals of Probability 8 (4): 745–764. doi:10.1214/aop/1176994663. MR577313. Zbl 0434.60034. 
  2. ^ Diaconis, P.; Freedman, D. (1980). "De Finetti's theorem for Markov chains". Annals of Probability 8 (1): 115–130. doi:10.1214/aop/1176994828. MR556418. Zbl 0426.60064. 
  3. ^ Steffen Lauritzen (2007) Sufficiency, Partial Exchangeability, and Exponential Families
  4. ^ Persi Diaconis and Svante Janson (2008) "Graph Limits and Exchangeable Random Graphs",Rendiconti di Matematica, Ser. VII 28(1), 33–61.
  5. ^ Cameron Freer and Daniel Roy (2009) "Computable exchangeable sequences have computable de Finetti measures", Proceedings of the 5th Conference on Computability in Europe: Mathematical Theory and Computational Practice, Lecture Notes In Computer Science, Vol. 5635, pp. 218–231.
  6. ^ Claus Koestler, Roland Speicher (2009) "A noncommutative de Finetti theorem: invariance under quantum permutations is equivalent to freeness with amalgamation" , Commun. Math. Phys. 291, 473–490.

Wikimedia Foundation. 2010.

Игры ⚽ Нужна курсовая?

Look at other dictionaries:

  • De Finetti's theorem — In probability theory, de Finetti s theorem explains why exchangeable observations are conditionally independent given some (usually) unobservable quantity to which an epistemic probability distribution would then be assigned. It is named in… …   Wikipedia

  • Bruno de Finetti — (* 13. Juni 1906 in Innsbruck; † 20. Juli 1985 in Rom) war ein italienischer Mathematiker. Seine wichtigsten Resultate sind in der Statistik und Wahrscheinlichkeitstheorie anzusiedeln. Inhaltsverzeichnis 1 Leben 2 Werk …   Deutsch Wikipedia

  • Bruno de Finetti — Born 13 June 1906(1906 06 13) Innsbruck, Austria …   Wikipedia

  • De Finetti — usually refers to the Italian probabilist and statistician Bruno de Finetti, noted for the operational subjective conception of probability. His works include: de Finetti s theorem, which explains why exchangeable observations are conditionally… …   Wikipedia

  • De Finetti diagram — A de Finetti diagram. The curved line is the expected Hardy Weinberg frequency as a function of p. A de Finetti diagram is a ternary plot used in population genetics. It is named after the Italian statistician Bruno de Finetti (1906–1985) and is… …   Wikipedia

  • de Finetti, Bruno — (1906–1985) Italian mathematician and philosopher. Born in Innsbruck, and educated at Milan, de Finetti worked on many different branches of mathematics, in Milan, in Rome, and from 1931 in the Assicurazioni Generali di Trieste, as an actuary and …   Philosophy dictionary

  • representation theorem — The proof given by de Finetti that from the requirement that someone s degrees of confidence in propositions be coherent, in the sense of avoiding the possibility of a Dutch book, it can be derived that their beliefs correspond to the classical… …   Philosophy dictionary

  • Bayesian probability — Bayesian statistics Theory Bayesian probability Probability interpretations Bayes theorem Bayes rule · Bayes factor Bayesian inference Bayesian network Prior · Posterior · Likelihood …   Wikipedia

  • List of mathematics articles (D) — NOTOC D D distribution D module D D Agostino s K squared test D Alembert Euler condition D Alembert operator D Alembert s formula D Alembert s paradox D Alembert s principle Dagger category Dagger compact category Dagger symmetric monoidal… …   Wikipedia

  • List of probability topics — This is a list of probability topics, by Wikipedia page. It overlaps with the (alphabetical) list of statistical topics. There are also the list of probabilists and list of statisticians.General aspects*Probability *Randomness, Pseudorandomness,… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”