Fisher information metric

Fisher information metric

In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space.

It can be used to calculate the informational difference between measurements.It takes the form:

:g_{jk}=int frac{partial log p(x, heta)}{partial heta_j} frac{partial log p(x, heta)}{partial heta_k} p(x, heta)dx.

Substituting i = -ln(p) from information theory, the formula becomes:

:g_{jk}=int frac{partial i(x, heta)}{partial heta_j} frac{partial i(x, heta)}{partial heta_k} p(x, heta)dx.

Which can be thought of intuitively as: "The distance between two points on a statistical differential manifold is the amount of information between them, i.e. the informational difference between them."

An equivalent form of the above equation is:

:g_{jk}=int frac{partial^2 i(x, heta)}{partial heta_j partial heta_k} p(x, heta)dx=mathrm{E}left [ frac{partial^2 i(x, heta)}{partial heta_j partial heta_k} ight] .

ee also

*Cramér-Rao bound
*Fisher information
*Bures metric


* Shun'ichi Amari - "Differential-geometrical methods in statistics", Lecture notes in statistics, Springer-Verlag, Berlin, 1985.
* Shun'ichi Amari, Hiroshi Nagaoka - "Methods of information geometry", Translations of mathematical monographs; v. 191, American Mathematical Society, 2000.

Wikimedia Foundation. 2010.

Игры ⚽ Поможем сделать НИР

Look at other dictionaries:

  • Fisher information — In statistics and information theory, the Fisher information (denoted mathcal{I}( heta)) is the variance of the score. It is named in honor of its inventor, the statistician R.A. Fisher.DefinitionThe Fisher information is a way of measuring the… …   Wikipedia

  • Information geometry — In mathematics and especially in statistical inference, information geometry is the study of probability and information by way of differential geometry. It reached maturity through the work of Shun ichi Amari in the 1980s, with what is currently …   Wikipedia

  • Fisher kernel — In mathematics, the Fisher kernel, named in honour of Sir Ronald Fisher, is a kernel. It was introduced in 1998 by Tommi Jaakkola [ Exploiting Generative Models in Discriminative Classifiers (1998) [… …   Wikipedia

  • Information theory — Not to be confused with Information science. Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental… …   Wikipedia

  • List of mathematics articles (F) — NOTOC F F₄ F algebra F coalgebra F distribution F divergence Fσ set F space F test F theory F. and M. Riesz theorem F1 Score Faà di Bruno s formula Face (geometry) Face configuration Face diagonal Facet (mathematics) Facetting… …   Wikipedia

  • CMA-ES — stands for Covariance Matrix Adaptation Evolution Strategy. Evolution strategies (ES) are stochastic, derivative free methods for numerical optimization of non linear or non convex continuous optimization problems. They belong to the class of… …   Wikipedia

  • Differential geometry — A triangle immersed in a saddle shape plane (a hyperbolic paraboloid), as well as two diverging ultraparallel lines. Differential geometry is a mathematical discipline that uses the techniques of differential and integral calculus, as well as… …   Wikipedia

  • List of differential geometry topics — This is a list of differential geometry topics. See also glossary of differential and metric geometry and list of Lie group topics. Contents 1 Differential geometry of curves and surfaces 1.1 Differential geometry of curves 1.2 Differential… …   Wikipedia

  • Divergence (statistics) — In statistics and information geometry, divergence or a contrast function is a function which establishes the “distance” of one probability distribution to the other on a statistical manifold. The divergence is a weaker notion than that of the… …   Wikipedia

  • Item response theory — In psychometrics, item response theory (IRT) is a body of theory describing the application of mathematical models to data from questionnaires and tests as a basis for measuring abilities, attitudes, or other variables. It is used for statistical …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”