Matthews Correlation Coefficient

The Matthews Correlation Coefficient is used in machine learning as a measure of the quality of binary (two class) classifications. It takes into account true and false positives and negatives and is generally regarded as a balanced measure which can be used even if the classes are of very different sizes. It returns a value between -1 and +1. A coefficient of +1 represents a perfect prediction, 0 an average random prediction and -1 an inverse prediction.

While there is no perfect way of describing the confusion matrix of true and false positives and negatives by a single number, the Matthews Correlation Coefficient is generally regarded as being one of the best such measures.

Other measures, such as the proportion of correct predictions (also termed Accuracy), are not useful when the two classes are of very different sizes. For example, assigning every object to the larger set achieves a high proportion of correct predictions, but is not generally a useful classification.

$MCC\; =\; frac\{\; TP\; imes\; TN\; -\; FP\; imes\; FN\; \}\; \{sqrt\{\; (TP\; +\; FP)\; (\; TP\; +\; FN\; )\; (\; TN\; +\; FP\; )\; (\; TN\; +\; FN\; )\; \}\; \}$

In this equation, TP is the number of true positives, TN the number of true negatives, FP the number of false positives and FN the number of false negatives. If any of the four sums in the denominator is zero, the denominator can be arbitrarily set to one; this results in a Matthews Correlation Coefficient of zero, which can be shown to be the correct limiting value.

References

* Baldi, P; Brunak, S.; Chauvin, Y.; Andersen, C. A. F.; Nielsen, H. Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics 2000, 16, 412-424.
* Matthews, B.W., Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim. Biophys. Acta 1975, 405, 442-451