Semi-Markov process
Encyclopedia
A continuous-time stochastic process
is called a semi-Markov process or 'Markov renewal process' if the embedded jump chain (the discrete process registering what values the process takes) is a Markov chain
, and where the holding times (time between jumps) are random variable
s with any distribution, whose distribution function
may depend on the two states between which the move is made. A semi-Markov process where all the holding times are exponentially distributed
is called a continuous time Markov chain/process (CTMC).
A semi-Markov process is associated with and can be constructed from a pair of processes W = (X,Y), where X is a Markov chain with state space S and transition probability matrix P, whereas Y is a process for which Y(n) depends only on r = X(n − 1) and s = X(n), and whose distribution function is Frs.
The semi-Markov process Z is then the process that chooses its sites on S according to X(n), and that chooses the transition time from X(n − 1) to X(n) according to Y(n).
Since the properties of Y (such as mean transition time) may depend on which site X chooses next, the processes Z are in general not a Markov process. Yet, the associated process W(n) = (X(n),Y(n)) is a Markov process. Hence the name semi-Markov.
Stochastic process
In probability theory, a stochastic process , or sometimes random process, is the counterpart to a deterministic process...
is called a semi-Markov process or 'Markov renewal process' if the embedded jump chain (the discrete process registering what values the process takes) is a Markov chain
Markov chain
A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the...
, and where the holding times (time between jumps) are random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
s with any distribution, whose distribution function
Distribution function
In molecular kinetic theory in physics, a particle's distribution function is a function of seven variables, f, which gives the number of particles per unit volume in phase space. It is the number of particles per unit volume having approximately the velocity near the place and time...
may depend on the two states between which the move is made. A semi-Markov process where all the holding times are exponentially distributed
Exponential distribution
In probability theory and statistics, the exponential distribution is a family of continuous probability distributions. It describes the time between events in a Poisson process, i.e...
is called a continuous time Markov chain/process (CTMC).
A semi-Markov process is associated with and can be constructed from a pair of processes W = (X,Y), where X is a Markov chain with state space S and transition probability matrix P, whereas Y is a process for which Y(n) depends only on r = X(n − 1) and s = X(n), and whose distribution function is Frs.
The semi-Markov process Z is then the process that chooses its sites on S according to X(n), and that chooses the transition time from X(n − 1) to X(n) according to Y(n).
Since the properties of Y (such as mean transition time) may depend on which site X chooses next, the processes Z are in general not a Markov process. Yet, the associated process W(n) = (X(n),Y(n)) is a Markov process. Hence the name semi-Markov.