de Moivre–Laplace theorem

de Moivre–Laplace theorem
As n grows large, the shape of the binomial distribution begins to resemble the smooth Gaussian curve.

In probability theory, the de Moivre–Laplace theorem is a normal approximation to the binomial distribution. It is a special case of the central limit theorem. It states that the binomial distribution of the number of "successes" in n independent Bernoulli trials with probability p of success on each trial is approximately a normal distribution with mean np and standard deviation \sqrt{npq}, if n is very large and some conditions are satisfied.

The theorem appeared in the second edition of The Doctrine of Chances by Abraham de Moivre, published in 1738. The "Bernoulli trials" were not so-called in that book, but rather de Moivre wrote about the probability distribution of the number of times "heads" appears when a coin is tossed 3600 times.[1]

Theorem

As n grows large, for k in the neighborhood of np we can approximate [2][3]

{n \choose k}\, p^k q^{n-k} \simeq \frac{1}{\sqrt{2 \pi npq}}\,e^{-(k-np)^2 / (2npq)}, \quad p+q=1,\ p>0,\ q>0

in the sense that the ratio of the left-hand side to the right-hand side converges to 1 as n\to\infty.

Proof

According to Stirling's formula, we can replace the factorial of a large number n, with the approximation:

n! \simeq \sqrt{2 \pi n} \left(\frac{n}{e}\right)^n \text{ , as n} \rightarrow \infty

or:

n! \simeq n^n e^{-n}\sqrt{2 \pi n} \text{ , as n} \rightarrow \infty.

Thus


\begin{align}
{n \choose k}\, p^k q^{n-k} & = \frac{n!}{k!\left(n-k\right)!} p^k q^{n-k} \\
& \simeq \frac{n^ne^{-n}\sqrt{2\pi n} }{k^ke^{-k}\sqrt{2\pi k} {(n-k)}^{n-k}e^{-(n-k)}\sqrt{2\pi (n-k)} }p^k q^{n-k}\\
& =\left[\frac{\sqrt{2\pi n} }{\sqrt{2\pi k} \sqrt{2\pi (n-k)} }\right]\left[\frac{n^n }{k^k {(n-k)}^{n-k} }\right]\left[\frac{e^{-n}}{e^{-k}e^{-(n-k)} }\right]p^k q^{n-k}\\
& =\left[\frac{\sqrt{n} }{\sqrt{k} \sqrt{2\pi (n-k)} }\right]\left[\frac{n^n }{k^k {(n-k)}^{n-k} }\right]\left[\frac{e^{-n}}{e^{-k} e^{-n}{ e}^k }\right]p^ kq^{n-k}\\
& =\left[\sqrt{\frac{n}{2\pi k(n-k)}}\right]\left[n^n{\left(\frac{p }{k }\right)}^k{\left(\frac{q }{n-k }\right)}^{(n- k)}\right] e^{-n+k+n-k}\\
& =\left[\sqrt{\frac{n}{2\pi k(n-k)}}\right]\left[n^{n-k+k}{\left(\frac{p }{k }\right)}^k{\left(\frac{q }{n-k }\right)}^{(n- k)}\right]\\
& =\left[\sqrt{\frac{n}{2\pi k(n-k)}}\right] \left[ n^{n-k} n^k {\left(\frac{p}{k}\right)}^k {\left(\frac{q}{n-k}\right)}^{(n-k)}\right]\\
& =\left[\sqrt{\frac{n}{2\pi k(n-k)}}\right]\left[{\left(\frac{np }{k }\right)}^k{\left(\frac{nq }{n-k }\right)}^{(n- k)} \right]\\
& =\left[\sqrt{\frac{n}{2\pi k(n-k)}}\right]\left[{\left(\frac{k }{np }\right)}^{-k}{\left(\frac{n-k}{nq }\right)}^{-(n- k)}\right]\\
\end{align}

Now, let

x=\frac{(k-np)}{\sqrt{npq}}
\Rightarrow \ k=np+x\sqrt{npq }  and  n-k=nq- x\sqrt{npq }
\Rightarrow \ \frac{k}{np}=1+x\sqrt{\frac{q}{np}}  and  \frac{n-k}{nq}=1-x\sqrt{\frac{p}{nq}}
\Rightarrow\ {n \choose k}p^kq^{n-k}\simeq \left[\sqrt{\frac{n}{2{\mathbf \pi }k(n-k)}}\right]\left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-(n - k)}\right]

Now, consider the first square bracket term.


\begin{align}
\sqrt{\frac{n}{2\pi k\left(n-k\right)}} & =\sqrt{\frac{n}{2\pi k\left(n-k\right)}\times \frac{{1}/{n^2}}{{1}/{n^2}}} \\
& =\sqrt{\frac{{1}/{n}}{{2\pi k(n-k)}/{n^2}}} \\
& =\sqrt{\frac{{1}/{n}}{2\pi \frac{k}{n}\frac{(n-k)}{n}}}\\
& =\sqrt{\frac{{1}/{n}}{2\pi \frac{k}{n}\left(1-\frac{k}{n}\right)}} \\
& =\sqrt{\frac{{1}/{n}}{2\pi p\left(1-p\right)}} \qquad \qquad \qquad \left[\because k\to np\Rightarrow \frac{k}{n}\to p\right] \\
& =\sqrt{\frac{{1}/{n}}{2\pi pq}} \qquad \qquad \qquad \left[\because p+q=1\Rightarrow q=1-p\right]\\
& =\sqrt{\frac{1}{2\pi npq}}\\
& =\frac{1}{\sqrt{2\pi npq}}\\
\end{align}
\Rightarrow{n \choose k}p^kq^{n-k}\simeq \frac{1}{\sqrt{2{\mathbf \pi }npq}}\left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-(n - k)}\right]

Now, consider the term in the second square bracket.

\left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n - k\right)}\right]=e^{\ln \left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n - k\right)}\right]}

The above conversion is based on the fact that the natural logarithm function, if considered as a real-valued function of a real variable, is the inverse function of the exponential function, leading to the identity given below:

eln(y) = y
\Rightarrow {n \choose k}p^kq^{n-k}\simeq \frac{1}{\sqrt{2{\mathbf \pi }npq}}e^{\ln \left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n - k\right)}\right]}

Now, consider the natural logarithm term.


\begin{align}
\ln\left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n-k\right)}\right] & =\ln{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}+\ln{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n-k\right)}\\
& =-k\ln\left(1+x\sqrt{\frac{q}{np}}\right)-\left(n-k\right)\ln\left(1-x\sqrt{\frac{p}{nq}}\right)\\
\end{align}

Now, using the following two series we will solve the above natural logarithm:-

\ln\left(1+y\right)=y-\frac{y^2}{2}+\frac{y^3}{3}-\frac{y^4}{4}+\cdots
\ln\left(1-y\right)=-y-\frac{y^2}{2}-\frac{y^3}{3}-\frac{y^4}{4}-\cdots
\Rightarrow \ln\left(1+x\sqrt{\frac{q}{np}}\right)=x\sqrt{\frac{q}{np}}-\frac{x^2q}{2np}+\cdots

and

\ln\left(1-x\sqrt{\frac{p}{nq}}\right)=-x\sqrt{\frac{p}{nq}}-\frac{x^2p}{2nq}-\cdots

\begin{align}
\Rightarrow{\ln \left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n-k\right)}\right]}& =-k\left(x\sqrt{\frac{q}{np}}-\frac{x^2q}{2np}+\cdots \right)-\left(n-k\right)\left(-x\sqrt{\frac{p}{nq}}-\frac{x^2p}{2nq}-\cdots \right)\\
& =-\left(np+x\sqrt{npq}\right)\left(x\sqrt{\frac{q}{np}}-\frac{x^2q}{2np}+\cdots \right)\\
& -\left(nq-x\sqrt{npq}\right)\left(-x\sqrt{\frac{p}{nq}}-\frac{x^2p}{2nq}-\cdots \right)\\
\end{align}

because

k=np+x\sqrt{npq}
n-k=nq-x\sqrt{npq}

and so


\begin{align}
\Rightarrow {\ln \left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n-k\right)}\right]}& =-\left(np\times x\sqrt{\frac{q}{np}}-np\times \frac{x^2q}{2np}+x\sqrt{npq}\times x\sqrt{\frac{q}{np}}-x\sqrt{npq}\times \frac{x^2q}{2np}+\cdots \right)\\
& -\left(-nq\times x\sqrt{\frac{p}{nq}}-nq\times \frac{x^2p}{2nq}+x\sqrt{npq}\times x\sqrt{\frac{p}{nq}}+x\sqrt{npq}\times \frac{x^2p}{2nq}+\cdots \right)\\
& =-\left(x\sqrt{npq}-\frac{x^2q}{2}+x^2q+\cdots \right)-\left(-x\sqrt{npq}-\frac{x^2p}{2}+x^2p+\cdots \right)\\
& =-\left(x\sqrt{npq}+\frac{x^2q}{2}+\cdots \right)-\left(-x\sqrt{npq}+\frac{x^2p}{2}+\cdots \right)\\
& =-x\sqrt{npq}-\frac{x^2q}{2}+x\sqrt{npq}-\frac{x^2p}{2}-\cdots \\
& =-\frac{x^2q}{2}-\frac{x^2p}{2}-\cdots \\
& =-\frac{x^2}{2}\left(q+p\right)-\cdots \\
& =-\frac{x^2}{2}-\cdots \\
\end{align}
\Rightarrow {\ln \left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n-k\right)}\right]} \simeq -\frac{x^2}{2}
\Rightarrow {n \choose k}p^kq^{n-k}\simeq \frac{1}{\sqrt{2{\mathbf \pi }npq}}e^{\ln\left[{\left(1+x\sqrt{\frac{q}{np}}\right)}^{-k}{\left(1-x\sqrt{\frac{p}{nq}}\right)}^{-\left(n-k\right)}\right]}\simeq \frac{1}{\sqrt{2{\mathbf \pi }npq}}e^{{-x^2}/{2}}

We can ignore the terms where the power of x is greater than 3 in the above expansion, since x approaches 0 as n gets large, because x is proportional to (knp), and knp.

x = \frac{ (k-np) }{ \sqrt{npq} }

Now,

x = \frac{ (k-np) }{ \sqrt{npq} }
\Rightarrow \frac{x^2}{2}=\frac{{\left(\frac{(k-np)}{\sqrt{npq}}\right)}^2}{2}=\frac{{\left(k-np\right)}^2}{2npq}

Thus,

{n \choose k}p^kq^{n-k}\simeq \frac{1}{\sqrt{2{\mathbf \pi }npq}}e^{{-{\left(k-np\right)}^2}/{2npq}}

and the result is proved.

Notes

  1. ^ Walker, Helen M (1985). "De Moivre on the law of normal probability". In Smith, David Eugene. A source book in mathematics. Dover. p. 78. ISBN 0486646904. http://www.york.ac.uk/depts/maths/histstat/demoivre.pdf. "But altho’ the taking an infinite number of Experiments be not practicable, yet the preceding Conclusions may very well be applied to finite numbers, provided they be great, for Instance, if 3600 Experiments be taken, make n = 3600, hence ½n will be = 1800, and ½√n 30, then the Probability of the Event’s neither appearing oftner than 1830 times, nor more rarely than 1770, will be 0.682688." 
  2. ^ Papoulis, Pillai, "Probability, Random Variables, and Stochastic Processes", 4th Edition
  3. ^ Feller, W. (1968) An Introduction to Probability Theory and Its Applications (Volume 1). Wiley. ISBN 0-471-25708-7. Section VII.3

Wikimedia Foundation. 2010.

Игры ⚽ Нужно сделать НИР?

Look at other dictionaries:

  • De Moivre–Laplace theorem — In probability theory, the de Moivre–Laplace theorem is a normal approximation to the binomial distribution. It is a special case of the central limit theorem. It states that the binomial distribution of the number of successes in n independent… …   Wikipedia

  • de Moivre's theorem — may be: de Moivre s formula – a trigonometric identity Theorem of de Moivre–Laplace – a central limit theorem This disambiguation page lists articles associated with the same title. If an internal link led you h …   Wikipedia

  • De Moivre's theorem — The de Moivres Theorem may refer to: *de Moivre s formula a trigonometric identity *Theorem of de Moivre–Laplace a central limit theorem …   Wikipedia

  • Abraham de Moivre — Moivre redirects here; for the French commune see Moivre, Marne. Abraham de Moivre Abraham de Moivre Born …   Wikipedia

  • Central limit theorem — This figure demonstrates the central limit theorem. The sample means are generated using a random number generator, which draws numbers between 1 and 100 from a uniform probability distribution. It illustrates that increasing sample sizes result… …   Wikipedia

  • Poisson limit theorem — The Poisson theorem gives a Poisson approximation to the binomial distribution, under certain conditions. [Papoulis, Pillai, Probability, Random Variables, and Stochastic Processes , 4th Edition] The theorem If:n ightarrow infty, p ightarrow 0,… …   Wikipedia

  • List of mathematics articles (D) — NOTOC D D distribution D module D D Agostino s K squared test D Alembert Euler condition D Alembert operator D Alembert s formula D Alembert s paradox D Alembert s principle Dagger category Dagger compact category Dagger symmetric monoidal… …   Wikipedia

  • List of important publications in statistics — Probability Théorie analytique des probabilités :Author: Pierre Simon Laplace:Publication data: 1820 (3rd ed.):Online version: ?:Description: Attacks the roots of least squares and interpolation techniques, bringing back techniques from a century …   Wikipedia

  • Geschichte der Stochastik — Roulettespieler, um 1800. Das Glücksspiel war eine der frühesten Triebfedern der Wahrscheinlichkeitsrechnung …   Deutsch Wikipedia

  • Geschichte der Wahrscheinlichkeitstheorie — Roulettespieler, um 1800. Das Glücksspiel war eine der frühesten Triebfedern der Wahrscheinlichkeitsrechnung …   Deutsch Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”