Markov property

In probability theory and statistics, the term Markov property refers to the memoryless property of a stochastic process. It was named after the Russian mathematician Andrey Markov.^[1]

A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it. A process with this property is called a Markov process. The term strong Markov property is similar to the Markov property, except that the meaning of "present" is defined in terms of a random variable known as a stopping time.

The term Markov assumption is used to describe a model where the Markov property is assumed to hold, such as a hidden Markov model.

A Markov random field,^[2] extends this property to two or more dimensions, or to random variables defined for an interconnected network of items. An example of a model for such a field is the Ising model.

For discrete-time processes with the Markov property, see Markov chain.

Both the terms "Markov property" and "strong Markov property" have been used in connection with a particular "memoryless" property of the exponential distribution.^[3]

Introduction

A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the present state; that is, given the present, the future does not depend on the past. A process with this property is said to be Markovian or a Markov process. The most famous Markov process is a Markov chain. Brownian motion is another well-known Markov process.

History

For some details of the early history of the Markov property see this brief account.

Definition

The following definition applies.^[4] An $\mathbb{R}^n$-valued stochastic process $X=(X_t:t\in I)$ on a probability space $(\Omega,\mathcal{F},\mathbb{P})$ is said to possess the Markov property if, for each $A \in \mathcal{B}(\mathbb{R}^n)$ and $s,t\in I, \, s<t$,

$\mathbb{P}(X_t \in A |\mathcal{F}_s)=\mathbb{P}(X_t \in A| \sigma(X_s)) ,$

where $\{\mathcal{F}\}_{t\in I}$ is the natural filtration and $\mathcal{B}(\mathbb{R}^n)$ denotes the Borel sigma-algebra on $\mathbb{R}^n$.

In the case that the process takes discrete values and is indexed by a discrete time, this can be reformulated as follows;

$\mathbb{P}(X_n=x_n|X_{n-1}=x_{n-1} \dots X_0=x_0)=\mathbb{P}(X_n=x_n|X_{n-1}=x_{n-1})$.

Strong Markov Property

Suppose that $X=(X_t:t\geq 0)$ is a stochastic process on a probability space $(\Omega,\mathcal{F},\mathbb{P})$ with natural filtration $\{\mathcal{F}_t\}_{t\geq 0}$. Then X is said to have the strong Markov property if, for each stopping time τ, conditioned on the event $\{\tau < \infty\}$, the process $X_{\tau + \cdot}$ (which maybe needs to be defined) is independent from $\mathcal{F}_{\tau}:=\{A \in \mathcal{F}: \tau \cap A \in \mathcal{F}_t ,\, \ t \geq 0\}$ and X_{τ + t} − X_τ has the same distribution as X_t for each $t \geq 0$.

The strong Markov property is a stronger property than the ordinary Markov property, since by taking the stopping time τ = t, the ordinary Markov property can be deduced.

Alternative Formulations

Alternatively, the Markov property can be formulated as follows;

$\mathbb{E}[f(X_t)|\mathcal{F}_s]=\mathbb{E}[f(X_t)|\sigma(X_s)]$

for all $t\geq s\geq 0$ and $f:\mathbb{R}^n\rightarrow \mathbb{R}$ bounded and measurable.

Applications

A very important application of the Markov property in a generalized form is in Markov chain Monte Carlo computations in the context of Bayesian statistics.

See also

• Markov chain
• Markov blanket
• Markov decision process
• Causal Markov condition
• Markov model
• Chapman–Kolmogorov equation

Notes

1. ^ Markov, A. A. (1954). Theory of Algorithms. [Translated by Jacques J. Schorr-Kon and PST staff] Imprint Moscow, Academy of Sciences of the USSR, 1954 [Jerusalem, Israel Program for Scientific Translations, 1961; available from Office of Technical Services, United States Department of Commerce] Added t.p. in Russian Translation of Works of the Mathematical Institute, Academy of Sciences of the USSR, v. 42. Original title: Teoriya algorifmov. [QA248.M2943 Dartmouth College library. U.S. Dept. of Commerce, Office of Technical Services, number OTS 60-51085.]
2. ^ Dodge, Y. (2003) The Oxford Dictionary of Statistical Terms, OUP. ISBN 0-19-850994-4
3. ^ Feller, W. (1971) Introduction to Probability Theory and Its Applications, Vol II (2nd edition),Wiley. ISBN 0-471-25709-5 (pages 9 and 20)
4. ^ Pascucci, Andrea. (2011) PDE and Martingale Methods in Option Pricing. Berlin: Springer, 2011. ISBN 8847017807