Fisher information metric

In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space.

It can be used to calculate the informational difference between measurements.It takes the form:

:$g\_\{jk\}=int\; frac\{partial\; log\; p(x,\; heta)\}\{partial\; heta\_j\}\; frac\{partial\; log\; p(x,\; heta)\}\{partial\; heta\_k\}\; p(x,\; heta)dx.$

Substituting $i\; =\; -ln(p)$ from information theory, the formula becomes:

:$g\_\{jk\}=int\; frac\{partial\; i(x,\; heta)\}\{partial\; heta\_j\}\; frac\{partial\; i(x,\; heta)\}\{partial\; heta\_k\}\; p(x,\; heta)dx.$

Which can be thought of intuitively as: "The distance between two points on a statistical differential manifold is the amount of information between them, i.e. the informational difference between them."

An equivalent form of the above equation is:

:$g\_\{jk\}=int\; frac\{partial^2\; i(x,\; heta)\}\{partial\; heta\_j\; partial\; heta\_k\}\; p(x,\; heta)dx=mathrm\{E\}left\; [\; frac\{partial^2\; i(x,\; heta)\}\{partial\; heta\_j\; partial\; heta\_k\}\; ight]\; .$

ee also

*Cramér-Rao bound
*Fisher information
*Bures metric

References

* Shun'ichi Amari - "Differential-geometrical methods in statistics", Lecture notes in statistics, Springer-Verlag, Berlin, 1985.
* Shun'ichi Amari, Hiroshi Nagaoka - "Methods of information geometry", Translations of mathematical monographs; v. 191, American Mathematical Society, 2000.