Entropy power inequality

In mathematics, the entropy power inequality is a result in probability theory that relates to so-called "entropy power" of random variables. It shows that the entropy power of suitably well-behaved random variables is a superadditive function. The entropy power inequality was proved in 1948 by Claude Shannon in his seminal paper "A Mathematical Theory of Communication". Shannon also provided a sufficient condition for equality to hold; Stam (1959) showed that the condition is in fact necessary.

tatement of the inequality

For a random variable "X" : Ω → R^"n" with probability density function "f" : R^"n" → R, the information entropy of "X", denoted "h"("X"), is defined to be

:$h(X)\; =\; -\; int\_\{mathbb\{R\}^\{n\; f(x)\; log\; f(x)\; ,\; mathrm\{d\}\; x$

and the entropy power of "X", denoted "N"("X"), is defined to be

:$N(X)\; =\; frac1\{2\; pi\; e\}\; exp\; left(\; frac\{2\}\{n\}\; h(X)\; ight).$

In particular,"N"("X") = |"K"| ^1/n when "X" ~ "Φ"_K.

Let "X" and "Y" be independent random variables with probability density functions in the "L"^"p" space "L"^"p"(R^"n") for some "p" > 1. Then

:$N(X\; +\; Y)\; geq\; N(X)\; +\; N(Y).\; ,$

Moreover, equality holds if and only if "X" and "Y" are multivariate normal random variables with proportional covariance matrices.

References

* cite journal
last = Dembo
first = Amir
coauthors = Cover, Thomas M. and Thomas, Joy A.
title = Information-theoretic inequalities
journal = IEEE Trans. Inform. Theory
volume = 37
year = 1991
issue = 6
pages = 1501–1518
issn = 0018-9448
doi = 10.1109/18.104312 MathSciNet|id=1134291
* cite journal
last=Gardner
first=Richard J.
title=The Brunn-Minkowski inequality
journal=Bull. Amer. Math. Soc. (N.S.)
volume=39
issue=3
year=2002
pages=355–405 (electronic)
doi=10.1090/S0273-0979-02-00941-2
* cite journal
last = Shannon
first = Claude E.
authorlink = Claude Shannon
title = A mathematical theory of communication
journal = Bell System Tech. J.
volume = 27
year = 1948
pages = 379–423, 623–656
* cite journal
last = Stam
first = A.J.
title = Some inequalities satisfied by the quantities of information of Fisher and Shannon
journal = Information and Control
volume = 2
year = 1959
pages = 101–112
doi = 10.1016/S0019-9958(59)90348-1