Cook's distance

In statistics, Cook's distance is a commonly used estimate of the influence of a data point when doing least squares regression analysis. In a practical ordinary least squares analysis, Cook's distance can be used in several ways: to indicate data points that are particularly worth checking for validity; to indicate regions of the design space where it would be good to be able obtain more data points.

Definition

Cook's distance measures the effect of deleting a given observation. Data points with large residuals (outliers) and/or high leverage may distort the outcome and accuracy of a regression. Points with a large Cook's distance are considered to merit closer examination in the analysis.

$D_i = \frac{ \sum_{j=1}^n (\hat Y_j\ - \hat Y_{j(i)})^2 }{p \ \mathrm{MSE}} .$

The following is an algebraically equivalent expression

$D_i = \frac{e_i^2}{p \ \mathrm{MSE}}\left[\frac{h_{ii}}{(1-h_{ii})^2}\right] .$

In the above equations:

$\hat Y_j \,$ is the prediction from the full regression model for observation j;

$\hat Y_{j(i)}\,$ is the prediction for observation j from a refitted regression model in which observation i has been omitted;

$h_{ii} \,$ is the i-th diagonal element of the hat matrix $\mathbf{X}\left(\mathbf{X}^T\mathbf{X}\right)^{-1}\mathbf{X}^T$;

$e_i \,$ is the crude residual (i.e., the difference between the observed value and the value fitted by the proposed model);

MSE is the mean square error of the regression model;

p is the number of fitted parameters in the model

Detecting highly influential observations using Cook's distance

There are different opinions regarding what cut-off values to use for spotting highly influential points. A simple operational guideline of D_i > 1 has been suggested.^[1] Others have indicated that D_i > 4 / n, where n is the number of observations, might be used.^[2]

Interpreting Cook's distance

Specifically D_i can be interpreted as the distance one's estimates move within the confidence ellipsoid that represents a region of plausible values for the parameters.^{[clarification needed]} This is shown by an alternative but equivalent representation of Cook's distance in terms of changes to the estimates of the regression parameters between the cases where the particular observation is either included or excluded from the regression analysis.

See also

• Outlier
• Leverage (statistics)
• Partial leverage
• DFFITS
• Studentized residual

References

1. ^ Cook, R. D. & Weisberg, S. (1982). Residuals and influence in regression. New York: Chapman & Hall.
2. ^ Bollen, K. A. & Jackman, R. (1990). Regression diagnostics: An expository treatment of outliers and influential cases. In: J. Fox & J. Scott Long (eds.) Modern Methods of Data Analysis (pp. 257-91). Newbury Park: Sage.

• Cook, R. Dennis (Feb 1977). "Detection of Influential Observations in Linear Regression". Technometrics (American Statistical Association) 19 (1): 15–18. doi:10.2307/1268249. JSTOR 1268249. MR0436478. 
• Cook, R. Dennis (Mar 1979). "Influential Observations in Linear Regression". Journal of the American Statistical Association (American Statistical Association) 74 (365): 169–174. doi:10.2307/2286747. JSTOR 2286747. MR0529533. 
• Lorenz, Frederick O. (Apr 1987). "Teaching about Influence in Simple Regression". Teaching Sociology (American Sociological Association) 15 (2): 173–177. doi:10.2307/1318032. JSTOR 1318032. 
• Chatterjee, S.; Hadi, A.S (2006). Regression analysis by example. John Wiley and Sons. ISBN 0471746967. 

External links

• Procedure for calculating Cook's distance