Zipf–Mandelbrot law

Probability distribution
name =Zipf–Mandelbrot
type =mass
pdf_

cdf_

parameters =$$N in {1,2,3ldots} (integer)
$$q in [0;infty) (real)
$$s>0, (real)
support =$$k in {1,2,ldots,N}
pdf =$$frac{1/(k+q)^s}{H_{N,q,s
cdf =$$frac{H_{k,q,s{H_{N,q,s
mean =$$frac{H_{N,q,s-1{H_{N,q,s-q
median =
mode =$$1,
variance =
skewness =
kurtosis =
entropy =
mgf =
char =
In probability theory and statistics, the Zipf–Mandelbrot law is a discrete probability distribution. Also known as the Pareto-Zipf law, it is a power-law distribution on ranked data, named after the linguist George Kingsley Zipf who suggested a simpler distribution called Zipf's law, and the mathematician Benoît Mandelbrot, who subsequently generalized it.

The probability mass function is given by:

:$$f(k;N,q,s)=frac{1/(k+q)^s}{H_{N,q,s

where $$H_{N,q,s} is given by:

:$$H_{N,q,s}=sum_{i=1}^N frac{1}{(i+q)^s}

which may be thought of as a generalization of a harmonic number. In the limit as $$N approaches infinity, this becomes the Hurwitz zeta function $$zeta(q,s). For finite $$N and $$q=0 the Zipf–Mandelbrot law becomes Zipf's law. For infinite $$N and $$q=0 it becomes a Zeta distribution.

Applications

The distribution of words ranked by their frequency in a random
text corpus is generally a power-law distribution, knownas Zipf's law.

If one plots the frequency rank of words contained in a large
corpus of text data versus the number of occurrences or actual
frequencies, one obtains a power-law distribution, with exponent close to one (but see Gelbukh and Sidorov 2001).

References and links

* Cite book
author = B. Mandelbrot
chapter = Information Theory and Psycholinguistics
title = Scientific psychology
editor= B.B. Wolman and E. Nagel
year = 1965
publisher = Basic Books Reprinted as
** Cite book
author = B. Mandelbrot
chapter = Information Theory and Psycholinguistics
title = Language
editor= R.C. Oldfield and J.C. Marchall
year = 1968
origyear = 1965
publisher = Penguin Books
* [http://arxiv.org/abs/physics/9901035 Z. K. Silagadze: Citations and the Zipf-Mandelbrot's law]
* [http://www.nist.gov/dads/HTML/zipfslaw.html NIST: Zipf's law]
* [http://www.nslij-genetics.org/wli/zipf/index.html W. Li's References on Zipf's law]
* [http://www.gelbukh.com/CV/Publications/2001/CICLing-2001-Zipf.htm Gelbukh and Sidorov 2001: Zipf and Heaps Laws’ Coefficients Depend on Language]