Stein's unbiased risk estimate

In statistics, Stein's unbiased risk estimate (SURE) is an unbiased estimator of the mean-squared error of a given estimator, in a deterministic estimation scenario. In other words, it provides an indication of the accuracy of a given estimator. This is important since, in deterministic estimation, the true mean-squared error of an estimator generally depends on the value of the unknown parameter, and thus cannot be determined completely.

The technique is named after its discoverer, Charles Stein. cite journal|title=Estimation of the Mean of a Multivariate Normal Distribution|journal=The Annals of Statistics|date=Nov. 1981|first=Charles M.|last=Stein|coauthors=|volume=9|issue=6|pages=1135–1151|id= |url=http://links.jstor.org/sici?sici=0090-5364%28198111%299%3A6%3C1135%3AEOTMOA%3E2.0.CO%3B2-5|accessdate=2008-03-30|month=Nov|year=1981|doi=10.1214/aos/1176345632 ]

Formal statement

Let $heta\; in\; \{mathbb\; R\}^n$ be an unknown deterministic parameter and let $x$ be a measurement vector which is distributed normally with mean $heta$ and covariance $sigma^2\; I$. Suppose $h(x)$ is an estimator of $heta$ from $x$. Then, Stein's unbiased risk estimate is given by:$mathrm\{SURE\}(h)\; =\; |\; heta|^2\; +\; |h(x)|^2\; +\; 2\; sigma^2\; sum\_\{i=1\}^n\; frac\{partial\; h\_i\}\{partial\; x\_i\}\; -\; 2\; sigma^2\; sum\_\{i=1\}^n\; x\_i\; h\_i(x)$where $h\_i(x)$ is the $i$th component of the estimate, and $|cdot|$ is the Euclidean norm.

The importance of SURE is that it is an unbiased estimate of the mean-squared error (or squared error risk) of $h(x)$, i.e.:$E\; \{\; mathrm\{SURE\}(h)\; \}\; =\; mathrm\{MSE\}(h).,!$

Thus, minimizing SURE can be expected to minimize the MSE. Except for the first term in SURE, which is identical for all estimators, there is no dependence on the unknown parameter $heta$ in the expression for SURE above. Thus, it can be manipulated (e.g., to determine optimal estimation settings) without knowledge of $heta$.

Applications

A standard application of SURE is to choose a parametric form for an estimator, and then optimize the values of the parameters to minimize the risk estimate. This technique has been applied in several settings. For example, a variant of the James-Stein estimator can be derived by finding the optimal shrinkage estimator. The technique has also been used by Donoho and Johnstone to determine the optimal shrinkage factor in a wavelet denoising setting. [ cite journal|title=Adapting to Unknown Smoothness via Wavelet Shrinkage|journal=Journal of the American Statistical Association|date=Dec. 1995|first=David L.|last=Donoho|coauthors=Iain M. Johnstone|volume=90|issue=432|pages=1200–1244|id= |url=http://links.jstor.org/sici?sici=0162-1459%28199512%2990%3A432%3C1200%3AATUSVW%3E2.0.CO%3B2-K|accessdate=2008-03-30|month=Dec|year=1995|doi=10.2307/2291512 ]

References