 Mallows' Cp

In statistics, Mallows' C_{p},^{[1]} named for Colin L. Mallows, is used to assess the fit of a regression model that has been estimated using ordinary least squares. It is applied in the context of model selection, where a number of predictor variables are available for predicting some outcome, and the goal is to find the best model involving a subset of these predictors. For example, one may be interested in predicting by how much a particular cholesterollowering drug will lower a particular person's cholesterol level, based on the person's age, gender, weight, and various dietary and lifestyle factors.
Definition and properties
Mallows' C_{p} addresses the issue of overfitting, in which model selection statistics such as the residual sum of squares always get smaller as more variables are added to a model. Thus, if we aim to select the model giving the smallest residual sum of squares, the model including all variables would always be selected. The C_{p} statistic calculated on a sample of data estimates the mean squared prediction error (MSPE) as its population target
where is the fitted value from the regression model for the j^{th} case, E(Y_{j}  X_{j}) is the expected value for the j^{th} case, and σ^{2} is the error variance (assumed constant across the cases). The MSPE will not automatically get smaller as more variables are added. The optimum model under this criterion is a compromise influenced by the sample size, the effect sizes of the different predictors, and the degree of collinearity between them.
If P regressors are selected from a set of K > P, the C_{p} statistic for that particular set of regressors is defined as :
where
 is the error sum of squares for the model with P regressors,
 Y_{pi} is the predicted value of the i'th observation of Y from the P regressors,
 S^{2} is the residual mean square after regression on the complete set of K regressors and can be estimated by mean square error MSE,
 and N is the sample size.
Practical use
The C_{p} statistic is often used as a stopping rule for various forms of stepwise regression. Mallows proposed the statistic as a criterion for selecting among many alternative subset regressions. Under a model not suffering from appreciable lack of fit (bias), C_{p} has expectation nearly equal to P; otherwise the expectation is roughly P plus a positive bias term. Nevertheless, even though it has expectation greater than or equal to P, there is nothing to prevent C_{p} < P or even C_{p} < 0 in extreme cases. It is suggested that one should choose a subset that has C_{p} approaching P,^{[2]} from above, for a list of subsets ordered by increasing P. In practice, the positive bias can be adjusted for by selecting a model from the ordered list of subsets, such that C_{p} < 2P.
Since the samplebased C_{p} statistic is an estimate of the MSPE, using C_{p} for model selection does not completely guard against overfitting. For instance, it is possible that the selected model will be one in which the sample C_{p} was a particularly severe underestimate of the MSPE.
Model selection statistics such as C_{p} are generally not used blindly, but rather information about the field of application, the intended use of the model, and any known biases in the data are taken into account in the process of model selection.
References
 Hocking, R.R. (1976). "The Analysis and Selection of Variables in Linear Regression". Biometrics 32 (1): 1–50. doi:10.2307/2529336. JSTOR 2529336.
Categories: Regression analysis
 Regression variable selection
Wikimedia Foundation. 2010.