Empirical process

The study of empirical processes is a branch of mathematical statistics and a sub-area of probability theory. It is a generalization of the central limit theorem for empirical measures.

Definition

It is known that under certain conditions empirical measures $P\_n$ uniformly converge to the probability measure "P" (see Glivenko-Cantelli theorem). The theory of "Empirical processes" provides the rate of this convergence.

A centered and scaled version of the empirical measure is the signed measure:$G\_n(A)=sqrt\{n\}(P\_n(A)-P(A))$It induces map on measurable functions "f" given by

:$fmapsto\; G\_n\; f=sqrt\{n\}(P\_n-P)f=sqrt\{n\}left(frac\{1\}\{n\}sum\_\{i=1\}^n\; f(X\_i)-mathbb\{E\}f\; ight)$

By the central limit theorem, $G\_n(A)$ converges in distribution to a normal random variable "N(0,P(A)(1-P(A)))" for fixed measurable set "A". Similarly, for a fixed function "f", $G\_nf$ converges in distribution to a normal random variable $N(0,mathbb\{E\}(f-mathbb\{E\}f)^2)$, provided that $mathbb\{E\}f$ and $mathbb\{E\}f^2$ exist.

Definition: is called an "empirical process" indexed by $mathcal\{C\}$, a collection of measurable subsets of "S".: is called an "empirical process" indexed by $mathcal\{F\}$, a collection of measurable functions from "S" to $mathbb\{R\}$.

A significant result in the area of empirical processes is Donsker's theorem. It has led to a study of the "Donsker classes" such that empirical processes indexed by these classes converge weakly to a certain Gaussian process. It can be shown that the Donsker classes are Glivenko-Cantelli, the converse is not true in general.

Example

As an example, consider empirical distribution functions. For real-valued iid random variables $X\_1,X\_n,...$ they are given by

:$F\_n(x)=P\_n((-infty,x]\; )=P\_nI\_\{(-infty,x]\; \}.$

In this case, empirical processes are indexed by a class $mathcal\{C\}=\{(-infty,x]\; :xinmathbb\{R\}\}.$ It has been shown that $mathcal\{C\}$ is a Donsker class, in particular,:$sqrt\{n\}(F\_n(x)-F(x))$ converges weakly in $ell^infty(mathbb\{R\})$ to a Brownian bridge "B(F(x))".

References

* P. Billingsley, Probability and Measure, John Wiley and Sons, New York, third edition, 1995.
* M.D. Donsker, Justification and extension of Doob's heuristic approach to the Kolmogorov-Smirnov theorems, Annals of Mathematical Statistics, 23:277-281, 1952.
* R.M. Dudley, Central limit theorems for empirical measures, Annals of Probability, 6(6): 899-929, 1978.
* R.M. Dudley, Uniform Central Limit Theorems, Cambridge Studies in Advanced Mathematics, 63, Cambridge University Press, Cambridge, UK, 1999.
* M.R. Kosorok, Inroduction to Empirical Processes and Semiparametric Inference, Springer, New York, 2008.
* Aad W. van der Vaart and Jon A. Wellner,Weak Convergence and Empirical Processes: With Applications to Statistics, 2nd ed., Springer, 2000. ISBN 978-0387946405
* J. Wolfowitz, Generalization of the theorem of Glivenko-Cantelli. Annals of Mathematical Statistics, 25, 131-138, 1954.

External links

* [http://www.stat.yale.edu/~pollard/Iowa/ Empirical Processes: Theory and Applications] , by David Pollard, a textbook available online.
* [http://www.bios.unc.edu/~kosorok/current.pdf Introduction to Empirical Processes and Semiparametric Inference] , by Michael Kosorok, another textbook available online.