Concentration parameter

Concentration parameter

In probability theory and statistics, a concentration parameter is a special kind of numerical parameter of a parametric family of probability distributions. Concentration parameters occur in conjunction with distributions whose domain is a probability distribution, such as the symmetric Dirichlet distribution and the Dirichlet process.

The larger the value of the concentration parameter, the more evenly distributed is the resulting distribution (the more it tends towards the uniform distribution). The smaller the value of the concentration parameter, the more sparsely distributed is the resulting distribution, with all but a few parameters having a probability near zero (in other words, the more it tends towards a distribution concentrated on a single point, the degenerate distribution defined by the Dirac delta function).

In the case of a Dirichlet distribution, a concentration parameter of 1 results in all sets of probabilities being equally likely, i.e. in this case the Dirichlet distribution of dimension k is equivalent to a uniform distribution over a k-dimensional simplex. Note that this is not the same as what happens when the concentration parameter tends towards infinity. In the former case, all resulting distributions are equally likely (the distribution over distributions is uniform). In the latter case, only near-uniform distributions are likely (the distribution over distributions is highly peaked around the uniform distribution). Meanwhile, in the limit as the concentration parameter tends towards zero, only distributions with nearly all mass concentrated on one of their components are likely (the distribution over distributions is highly peaked around the k possible Dirac delta distributions centered on one of the components, or in terms of the k-dimensional simplex, is highly peaked at corners of the simplex).

An example of where a sparse prior (concentration parameter much less than 1) is called for, consider a topic model, which is used to learn the topics that are discussed in a set of documents, where each "topic" is described using a categorical distribution over a vocabulary of words. A typical vocabulary might have 100,000 words, leading to a 100,000-dimensional categorical distribution. The prior distribution for the parameters of the categorical distribution would likely be a symmetric Dirichlet distribution. However, a coherent topic might only have a few hundred words with any significant probability mass. Accordingly, a reasonable setting for the concentration parameter might be 0.01 or 0.001. With a larger vocabulary of around 1,000,000 words, an even smaller value, e.g. 0.0001, might be appropriate.

See also


Wikimedia Foundation. 2010.

Игры ⚽ Нужно сделать НИР?

Look at other dictionaries:

  • Concentration inequality — In mathematics, concentration inequalities provide probability bounds on how a random variable deviates from some value (e.g. its expectation). The laws of large numbers of classical probability theory state that sums of independent random… …   Wikipedia

  • Parameter — For other uses, see Parameter (disambiguation). Parameter from Ancient Greek παρά also “para” meaning “beside, subsidiary” and μέτρον also “metron” meaning “measure”, can be interpreted in mathematics, logic, linguistics, environmental science[1] …   Wikipedia

  • Statistical parameter — A statistical parameter is a parameter that indexes a family of probability distributions. It can be regarded as a numerical characteristic of a population or a model.[1] Among parameterized families of distributions are the normal distributions …   Wikipedia

  • Measures of pollutant concentration — NOEL redirects here. For other uses, see Noel (disambiguation). Measures of pollutant concentration are used to determine risk assessment in public health. Industry is continually synthesizing new chemicals, the regulation of which requires… …   Wikipedia

  • No effect concentration — (NEC) is a risk assessment parameter that represents the concentration of a pollutant that will not harm the species involved, with respect to the effect that is studied. It is often the starting point for environmental policy [Bruijn et al, 1997 …   Wikipedia

  • Market concentration — In economics, market concentration is a function of the number of firms and their respective shares of the total production (alternatively, total capacity or total reserves) in a market. Alternative terms are Industry concentration and Seller… …   Wikipedia

  • Dirichlet distribution — Several images of the probability density of the Dirichlet distribution when K=3 for various parameter vectors α. Clockwise from top left: α=(6, 2, 2), (3, 7, 5), (6, 2, 6), (2, 3, 4). In probability and… …   Wikipedia

  • Dirichlet process — In probability theory, a Dirichlet process is a stochastic process that can be thought of as a probability distribution whose domain is itself a random distribution. That is, given a Dirichlet process , where H (the base distribution) is an… …   Wikipedia

  • Mixture model — See also: Mixture distribution In statistics, a mixture model is a probabilistic model for representing the presence of sub populations within an overall population, without requiring that an observed data set should identify the sub population… …   Wikipedia

  • Von Mises–Fisher distribution — The von Mises–Fisher distribution is a probability distribution on the (p 1) dimensional sphere in mathbb{R}^{p}. If p=2the distribution reduces to the von Mises distribution on the circle. The distributionbelongs to the field of directional… …   Wikipedia

Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”