Semi-Markov process

Semi-Markov process

A continuous-time stochastic process is called a semi-Markov process or 'Markov renewal process' if the embedded jump chain (the discrete process registering what values the process takes) is a Markov chain, and where the holding times (time between jumps) are random variables with any distribution, whose distribution function may depend on the two states between which the move is made. A semi-Markov process where all the holding times are exponentially distributed is called a continuous time Markov chain/process (CTMC).

A semi-Markov process is associated with and can be constructed from a pair of processes W = (X,Y), where X is a Markov chain with state space S and transition probability matrix P, whereas Y is a process for which Y(n) depends only on r = X(n − 1) and s = X(n), and whose distribution function is Frs[clarification needed].

The semi-Markov process Z is then the process that chooses its sites on S according to X(n), and that chooses the transition time from X(n − 1) to X(n) according to Y(n).

Since the properties of Y (such as mean transition time) may depend on which site X chooses next, the processes Z are in general not a Markov process. Yet, the associated process W(n) = (X(n),Y(n)) is a Markov process. Hence the name semi-Markov.

See also

References and Further Reading

  • Ross, S.M. (1995) Stochastic Processes. Wiley. ISBN 978-0471120629
  • Barbu, V.S, Limnios, N. (2008) Semi-Markov Chains and Hidden Semi-Markov Models toward Applications: Their Use in Reliability and DNA Analysis. ISBN 978-0387731711

Wikimedia Foundation. 2010.

Игры ⚽ Поможем сделать НИР

Look at other dictionaries:

  • Markov process — In probability theory and statistics, a Markov process, named after the Russian mathematician Andrey Markov, is a time varying random phenomenon for which a specific property (the Markov property) holds. In a common description, a stochastic… …   Wikipedia

  • Continuous-time Markov process — In probability theory, a continuous time Markov process is a stochastic process { X(t) : t ≥ 0 } that satisfies the Markov property and takes values from a set called the state space; it is the continuous time version of a Markov chain. The… …   Wikipedia

  • Hidden semi-Markov model — A hidden semi Markov model (HSMM) is a statistical model with the same structure as a hidden Markov model except that the unobservable process is semi Markov rather than Markov. This means that the probability of there being a change in the… …   Wikipedia

  • Markov chain — A simple two state Markov chain. A Markov chain, named for Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized …   Wikipedia

  • Variable-order Markov model — Variable order Markov (VOM) models are an important class of models that extend the well known Markov chain models. In contrast to the Markov chain models, where each random variable in a sequence with a Markov property depends on a fixed number… …   Wikipedia

  • Quasi-birth-death process — In queueing models, the quasi birth death process describes a generlisation of the birth death process. As with the birth death process movements between it moves skip free up and down, but unlike the birth death process the process is semi… …   Wikipedia

  • Hidden Markov model — Probabilistic parameters of a hidden Markov model (example) x mdash; states y mdash; possible observations a mdash; state transition probabilities b mdash; output probabilitiesA hidden Markov model (HMM) is a statistical model in which the system …   Wikipedia

  • Gauss–Markov theorem — This article is not about Gauss–Markov processes. In statistics, the Gauss–Markov theorem, named after Carl Friedrich Gauss and Andrey Markov, states that in a linear model in which the errors have expectation zero and are uncorrelated and have… …   Wikipedia

  • List of stochastic processes topics — In the mathematics of probability, a stochastic process can be thought of as a random function. In practical applications, the domain over which the function is defined is a time interval ( time series ) or a region of space ( random field… …   Wikipedia

  • Renewal theory — is the branch of probability theory that generalizes Poisson processes for arbitrary holding times . Applications include calculating the expected time for a monkey who is randomly tapping at a keyboard to type the word Macbeth and comparing the… …   Wikipedia

Share the article and excerpts

Direct link
… Do a right-click on the link above
and select “Copy Link”