Unbiased estimation of standard deviation

In statistics, the standard deviation is often estimated from a random sample drawn from the population. The most common measure used is the "sample standard deviation", which is defined by:$s\; =\; sqrt\{frac\{1\}\{n-1\}\; sum\_\{i=1\}^n\; (x\_i\; -\; overline\{x\})^2\},,$where $\{x\_1,x\_2,ldots,x\_n\}$ is the sample (formally, realizations from a random variable "X") and $overline\{x\}$ is the sample mean.

The reason for this definition is that "s"² is an unbiased estimator for the variance σ² of the underlying population, if that variance exists and the sample values are drawn independently with replacement. However, "s" estimates the population standard deviation σ with negative bias; that is, "s" tends to underestimate σ.

An explanation why the square root of the sample variance is a biased estimator of the standard deviation is that the square root is a nonlinear function, and only linear functions commute with taking the mean. Since the square root is a concave function, it follows from Jensen's inequality that the square root of the sample variance is an underestimate.

Bias correction

When the random variable is normally distributed, a minor correction exists to eliminate the bias. To derive the correction, note that for normally distributed "X", Cochran's theorem implies that $sqrt\{n\{-\}1\},s/sigma$ has a chi distribution with $n-1$ degrees of freedom. Consequently,:$operatorname\{E\}\; [s]\; =\; c\_4sigma$where $c\_4$ is a constant that depends on the sample size "n" as follows::$c\_4=sqrt\{frac\{2\}\{n-1frac\{Gammaleft(frac\{n\}\{2\}\; ight)\}\{Gammaleft(frac\{n-1\}\{2\}\; ight)\}\; =\; 1\; -\; frac\{1\}\{4n\}\; -\; frac\{7\}\{32n^2\}\; -\; O(n^\{-3\})$and $Gamma(cdot)$ is the gamma function.

Thus an unbiased estimator of σ is had by dividing "s" by $c\_4$. Tables giving the value of $c\_4$ for selected values of "n" may be found in most textbooks on statistical quality control. As "n" grows large it approaches 1, and even for smaller values the correction is minor. For example, for $n=2,5,10$ the values of $c\_4$ are about 0.7979, 0.9400, 0.9727. It is important to keep in mind this correction only produces an unbiased estimator for normally distributed "X". When this condition is satisfied, another result about "s" involving $c\_4$ is that the standard deviation of "s" is $sigmasqrt\{1-c\_4^\{2$, while the standard deviation of the unbiased estimator is $sigmasqrt\{c\_4^\{-2\}-1\}\; .$

ee also

*Estimation of covariance matrices
*Sample mean and sample covariance

References

* [http://www.itl.nist.gov/div898/handbook/pmc/section3/pmc32.htm What are Variables Control Charts?]
* Douglas C. Montgomery and George C. Runger, "Applied Statistics and Probability for Engineers", 3rd edition, Wiley and sons, 2003. (see Sections 7-2.2 and 16-5)