Zipf–Mandelbrot law

Probability distribution
name =Zipf–Mandelbrot
type =mass
pdf_

cdf_

parameters =$N\; in\; \{1,2,3ldots\}$ (integer)
$q\; in\; [0;infty)$ (real)
$s>0,$ (real)
support =$k\; in\; \{1,2,ldots,N\}$
pdf =$frac\{1/(k+q)^s\}\{H\_\{N,q,s$
cdf =$frac\{H\_\{k,q,s\{H\_\{N,q,s$
mean =$frac\{H\_\{N,q,s-1\{H\_\{N,q,s-q$
median =
mode =$1,$
variance =
skewness =
kurtosis =
entropy =
mgf =
char =
In probability theory and statistics, the Zipf–Mandelbrot law is a discrete probability distribution. Also known as the Pareto-Zipf law, it is a power-law distribution on ranked data, named after the linguist George Kingsley Zipf who suggested a simpler distribution called Zipf's law, and the mathematician Benoît Mandelbrot, who subsequently generalized it.

The probability mass function is given by:

:$f(k;N,q,s)=frac\{1/(k+q)^s\}\{H\_\{N,q,s$

where $H\_\{N,q,s\}$ is given by:

:$H\_\{N,q,s\}=sum\_\{i=1\}^N\; frac\{1\}\{(i+q)^s\}$

which may be thought of as a generalization of a harmonic number. In the limit as $N$ approaches infinity, this becomes the Hurwitz zeta function $zeta(q,s)$. For finite $N$ and $q=0$ the Zipf–Mandelbrot law becomes Zipf's law. For infinite $N$ and $q=0$ it becomes a Zeta distribution.

Applications

The distribution of words ranked by their frequency in a random
text corpus is generally a power-law distribution, knownas Zipf's law.

If one plots the frequency rank of words contained in a large
corpus of text data versus the number of occurrences or actual
frequencies, one obtains a power-law distribution, with exponent close to one (but see Gelbukh and Sidorov 2001).

References and links

* Cite book
author = B. Mandelbrot
chapter = Information Theory and Psycholinguistics
title = Scientific psychology
editor= B.B. Wolman and E. Nagel
year = 1965
publisher = Basic Books Reprinted as
** Cite book
author = B. Mandelbrot
chapter = Information Theory and Psycholinguistics
title = Language
editor= R.C. Oldfield and J.C. Marchall
year = 1968
origyear = 1965
publisher = Penguin Books
* [http://arxiv.org/abs/physics/9901035 Z. K. Silagadze: Citations and the Zipf-Mandelbrot's law]
* [http://www.nist.gov/dads/HTML/zipfslaw.html NIST: Zipf's law]
* [http://www.nslij-genetics.org/wli/zipf/index.html W. Li's References on Zipf's law]
* [http://www.gelbukh.com/CV/Publications/2001/CICLing-2001-Zipf.htm Gelbukh and Sidorov 2001: Zipf and Heaps Laws’ Coefficients Depend on Language]