Quantum relative entropy

In quantum information theory, quantum relative entropy is a measure of distinguishability between two quantum states. It is the quantum mechanical analog of relative entropy.

Motivation

For simplicity, it will be assumed that all objects in the article are finite dimensional.

We first discuss the classical case. Suppose the probabilities of a finite sequence of events is given by the probability distribution "P" = {"p"₁..."p"_n}, but somehow we mistakenly assumed it to be "Q" = {"q"₁..."q"_n}. For instance, we can mistake an unfair coin for a fair one. According to this erroneous assumption, our uncertainty about the "j"-th event, or equivalently, the amount of information provided after observing the "j"-th event, is

:$;\; -\; log\; q\_j.$

The (assumed) average uncertainty of all possible events is then

:$;\; -\; sum\_j\; p\_j\; log\; q\_j.$

On the other hand, the Shannon entropy of the probability distribution "p", defined by

:$;\; -\; sum\_j\; p\_j\; log\; p\_j,$

is the real amount of uncertainty before observation. Therefore the difference between these two quantities

:$;\; -\; sum\_j\; p\_j\; log\; q\_j\; -\; (-\; sum\_j\; p\_j\; log\; p\_j)\; =\; sum\_j\; p\_j\; log\; p\_j\; -\; sum\_j\; p\_j\; log\; q\_j$

is a measure of the distinguishability of the two probability distributions "p" and "q". This is precisely the classical relative entropy, or Kullback–Leibler divergence:

:$D\_\{mathrm\{KL(P|Q)\; =\; sum\_j\; p\_j\; log\; frac\{p\_j\}\{q\_j\}\; !.$

Note
#In the definitions above, the convention that 0·log 0 = 0 is assumed, since lim_{"x" → 0} "x" log "x" = 0. Intuitively, one would expect that an event of zero probability to contribute nothing towards entropy.
#The relative entropy is not a metric. For example, it is not symmetric. The uncertainty discrepancy in mistaking a fair coin to be unfair is not the same as the opposite situation.

Definition

As with many other objects in quantum information theory, quantum relative entropy is defined by extending the classical definition from probability distributions to density matrices. Let "ρ" be a density matrix. The von Neumann entropy of "ρ", which is the quantum mechanical analaog of the Shannon entropy, is given by

:$S(\; ho)\; =\; -\; operatorname\{Tr\}\; ho\; log\; ho.$

For two density matrices "ρ" and "σ", the quantum relative entropy of "ρ" with respect to "σ" is defined by

:$S(\; ho\; |\; sigma)\; =\; -\; operatorname\{Tr\}\; ho\; log\; sigma\; -\; S(\; ho)\; =\; operatorname\{Tr\}\; ho\; log\; ho\; -\; operatorname\{Tr\}\; ho\; log\; sigma\; =\; operatorname\{Tr\}\; ho\; (log\; ho\; -\; log\; sigma).$

We see that, when the states are classical, i.e. "ρσ" = "σρ", the definition coincides with the classical case.

Non-finite relative entropy

In general, the "support" of a matrix "M", denoted by "supp"("M"), is the orthogonal complement of its kernel. When consider the quantum relative entropy, we assume the convention that - "s"· log 0 = ∞ for any "s" > 0. This leads to the definition that

:$S(\; ho\; |\; sigma)\; =\; infty$

when

:$supp(\; ho)\; cap\; supp(sigma)^\{perp\}\; eq\; \{0\}.$

This makes physical sense. Informally, the quantum relative entropy is a measure of our ability to distinguish two quantum states. But orthogonal quantum states can always be distinguished, via projective measurement. In the present context, this is reflected by non-finite quantum relative entropy.

In the interpretation given in the previous section, if we erroneously assume the state "ρ" has support in"supp"("ρ")^⊥, this is an error impossible to recover from.

Klein's inequality

Corresponding classical statement

For the classical Kullback–Leibler divergence, it can be shown that

:$D\_\{mathrm\{KL(P|Q)\; =\; sum\_j\; p\_j\; log\; frac\{p\_j\}\{q\_j\}\; geq\; 0,$

and equality holds if and only if "P" = "Q". Colloquially, this means that the uncertainty calculated using erroneous assumptions is always greater than the real amount of uncertainty.

To show the inequality, we rewrite

:$D\_\{mathrm\{KL(P|Q)\; =\; sum\_j\; p\_j\; log\; frac\{p\_j\}\{q\_j\}\; =\; sum\_j\; (-\; log\; frac\{q\_j\}\{p\_j\})(p\_j).$

Notice that log is a concave function. Therefore -log is convex. Applying Jensen's inequality to -log gives

:$D\_\{mathrm\{KL(P|Q)\; =\; sum\_j\; (-\; log\; frac\{q\_j\}\{p\_j\})(p\_j)\; geq\; -\; log\; (\; sum\_j\; frac\{q\_j\}\{p\_j\}\; p\_j\; )\; =\; 0.$

Jensen's inequality also states that equality holds if and only if, for all "i", "q_i" = (∑"q_j") "p_i", i.e. "p" = "q".

The result

Klein's inequality states that the quantum relative entropy

:$S(\; ho\; |\; sigma)\; =\; operatorname\{Tr\}\; ho\; (log\; ho\; -\; log\; sigma).$

is non-negative in general. It is zero if and only "ρ" = "σ".

Proof

Let "ρ" and "σ" have spectral decompositions

:$ho\; =\; sum\_i\; p\_i\; v\_i\; v\_i\; ^*\; ;\; ,\; ;\; sigma\; =\; sum\_i\; q\_i\; w\_i\; w\_i\; ^*.$

So

:$log\; ho\; =\; sum\_i\; (log\; p\_i)\; v\_i\; v\_i\; ^*\; ;\; ,\; ;\; log\; sigma\; =\; sum\_i\; (log\; q\_i)w\_i\; w\_i\; ^*.$

Direct calculation gives

:$S(\; ho\; |\; sigma)$ :$=\; sum\_k\; p\_k\; log\; p\_k\; -\; sum\_\{i,j\}\; (p\_i\; log\; q\_j)\; |\; v\_i\; ^*\; w\_j\; |^2$:$=\; sum\_i\; p\_i\; (\; log\; p\_i\; -\; sum\_j\; log\; q\_j\; |\; v\_i\; ^*\; w\_j\; |^2)$:$;=\; sum\_i\; p\_i\; (log\; p\_i\; -\; sum\_j\; (log\; q\_j\; )P\_\{ij\}),$ where "P_{i j}" = |"v_i*w_j"|².

Since the matrix ("P_{i j}")"_{i j}" is a doubly stochastic matrix and -log is a concave function, the above expression is

:$geq\; sum\_i\; p\_i\; (log\; p\_i\; -\; log\; (sum\_j\; q\_j\; P\_\{ij\})$

:$;\; =\; sum\_i\; p\_i\; (log\; p\_i\; -\; log\; (sum\_j\; q\_j\; P\_\{ij\}).$

Define "r"_i = ∑_"j""q_j P_{i j}". Then {"r"_i} is a probability distribution. From the non-negativity of classical relative entropy, we have

:$S(\; ho\; |\; sigma)\; geq\; sum\_i\; p\_i\; log\; frac\{p\_i\}\{r\_i\}\; geq\; 0.$

The second part of the claim follows from the fact that, since -log is strictly convex, equality is achieved in

:$sum\_i\; p\_i\; (log\; p\_i\; -\; sum\_j\; (log\; q\_j\; )P\_\{ij\})\; geq\; sum\_i\; p\_i\; (log\; p\_i\; -\; log\; (sum\_j\; q\_j\; P\_\{ij\})$

if and only if ("P_{i j}") is a permutation matrix, which implies "ρ" = "σ", after a suitable labeling of the eigenvectors {"v_i"} and {"w_i"}.

An entanglement measure

Let a composite quantum system have state space

:$H\; =\; otimes\; \_k\; H\_k$

and "ρ" be a density matrix acting on "H".

The relative entropy of entanglement of "ρ" is defined by

:$;\; D\_\{mathrm\{REE\; (\; ho)\; =\; min\_\{sigma\}\; S(\; ho\; |\; sigma)$

where the minimum is taken over the family of separable states. A physical interpretation of the quantity is the optimal distinguishability of the state "ρ" from separable states.

Clearly, when "ρ" is not entangled

:$;\; D\_\{mathrm\{REE\; (\; ho)\; =\; 0$

by Klein's inequality.