 Method of moments (statistics)

See method of moments (probability theory) for an account of a technique for proving convergence in distribution.
In statistics, the method of moments is a method of estimation of population parameters such as mean, variance, median, etc. (which need not be moments), by equating sample moments with unobservable population moments and then solving those equations for the quantities to be estimated.
Contents
Methodology
Suppose that the problem is to estimate p unknown parameters θ_{1},θ_{2},...θ_{p} characterizing a distribution f_{W}(w;θ). Suppose p of the moments of the true distribution can be expressed as functions of the θs:
 ...
Let be the jth sample moment corresponding to the population moment μ_{j}. The method of moments estimator for θ_{1},θ_{2},...θ_{p} denoted by is defined by the solution (if there is one) to the equations:^{[citation needed]}
 ...
Example
Suppose X_{1}, ..., X_{n} are independent identically distributed random variables with a gamma distribution with probability density function
for x > 0, and 0 for x < 0.
The first moment, i.e., the expected value, of a random variable with this probability distribution is
and the second moment, i.e., the expected value of its square, is
These are the "population moments".
The first and second "sample moments" m_{1} and m_{2} are respectively
and
Equating the population moments with the sample moments, we get
and
Solving these two equations for α and β, we get
and
We then use these 2 quantities as estimates, based on the sample, of the two unobservable population parameters α and β.
Advantages and disadvantages of this method
In some respects, when estimating parameters of a known family of probability distributions, this method was superseded by Fisher's method of maximum likelihood, because maximum likelihood estimators have higher probability of being close to the quantities to be estimated.
However, in some cases, as in the above example of the gamma distribution, the likelihood equations may be intractable without computers, whereas the methodofmoments estimators can be quickly and easily calculated by hand as shown above.
Estimates by the method of moments may be used as the first approximation to the solutions of the likelihood equations, and successive improved approximations may then be found by the Newton–Raphson method. In this way the method of moments and the method of maximum likelihood are symbiotic.
In some cases, infrequent with large samples but not so infrequent with small samples, the estimates given by the method of moments are outside of the parameter space; it does not make sense to rely on them then. That problem never arises in the method of maximum likelihood. Also, estimates by the method of moments are not necessarily sufficient statistics, i.e., they sometimes fail to take into account all relevant information in the sample.
When estimating other structural parameters (e.g., parameters of a utility function, instead of parameters of a known probability distribution), appropriate probability distributions may not be known, and momentbased estimates may be preferred to MLE.
See also
Categories:  ...
Wikimedia Foundation. 2010.