Bhattacharyya distance

In statistics, the Bhattacharyya distance measures the similarity of two discrete probability distributions. It is normally used to measure the separability of classes in classification.

For discrete probability distributions p and q over the same domain X, it is defined as::$D\_B(p,q)\; =\; -ln\; left(\; sum\_\{xin\; X\}\; sqrt\{p(x)\; q(x)\}\; ight)$where::$BC(p,q)\; =\; sum\_\{xin\; X\}\; sqrt\{p(x)\; q(x)\}$is the Bhattacharyya coefficient.For continuous distributions, the Bhattacharyya coefficient is defined as::$BC(p,q)\; =\; int\; sqrt\{p(x)\; q(x)\},\; dx$In either case, $0\; le\; BC\; le\; 1$ and $0\; le\; D\_B\; le\; infty$. $D\_B$ need not obey the triangle inequality, but $sqrt\{1-BC\}$ does obey the triangle inequality.

For multivariate Gaussian distributions $p\_i=N(m\_i,P\_i)$,:$D\_B=\{1over\; 8\}(m\_1-m\_2)^T\; P^\{-1\}(m\_1-m\_2)+\{1over\; 2\}ln\; ,left(\{det\; P\; over\; sqrt\{det\; P\_1\; ,\; det\; P\_2\}\; \}\; ight)$,where $m\_i$ and $P\_i$ are the means and covariances of the distributions, and:$P=\{P\_1+P\_2\; over\; 2\}$. Note that the first term in the Bhattacharyya distance is related to the Mahalanobis distance.

ee also

* Hellinger distance
* Mahalanobis distance
* Chernoff bound

References

* A. Bhattacharyya, "On a measure of divergence between two statistical populations defined by probability distributions", "Bull. Calcutta Math. Soc.", vol. 35, pp. 99–109, 1943.
*T. Kailath, "The Divergence and Bhattacharyya Distance Measures in Signal Selection", "IEEE Trans. on Comm. Technology", vol. 15, pp. 52-60, Feb. 1967.
*A. Djouadi, O. Snorrason and F. Garber, "The quality of Training-Sample estimates of the Bhattacharyya coefficient", IEEE Tran. Pattern analysis and machine intelligence, vol. 12, pp. 92-97, 1990.

For a short list of properties, see: http://www.mtm.ufsc.br/~taneja/book/node20.html