In
mathematicsMathematics is the science and study of quantity, structure, space, and change. Mathematicians seek out patterns, formulate new conjectures, and establish truth by rigorous deduction from appropriately chosen axioms and definitions....
, the term
Markov property or
Markov-type property can refer to either of two closely-related things.
In the narrowest sense, a
stochastic processIn probability theory, a stochastic process, or sometimes random process, is the counterpart to a deterministic process...
has the
Markov property if the conditional probability distribution of future states of the process, given the present state and a constant number of past states, depend only upon the present state and the given states in the past, but not on any other past states,
i.e. it is conditionally independent of these older states.
In
mathematicsMathematics is the science and study of quantity, structure, space, and change. Mathematicians seek out patterns, formulate new conjectures, and establish truth by rigorous deduction from appropriately chosen axioms and definitions....
, the term
Markov property or
Markov-type property can refer to either of two closely-related things.
In the narrowest sense, a
stochastic processIn probability theory, a stochastic process, or sometimes random process, is the counterpart to a deterministic process...
has the
Markov property if the conditional probability distribution of future states of the process, given the present state and a constant number of past states, depend only upon the present state and the given states in the past, but not on any other past states,
i.e. it is conditionally independent of these older states. Such a process is called
Markovian or a
Markov processA Markov process, named after the Russian mathematician Andrey Markov, is a mathematical model for the random evolution of a memoryless system, that is, one for which the likelihood of a given future state, at any given moment, depends only on its present state, and not on any past states.In a...
. The articles
Markov chainIn mathematics, a Markov chain, named after Andrey Markov, is a random process where all information about the future is contained in the present state . To be more exact, the process has the Markov property, meaning that future states depend only on the present state, and are independent of past...
and
continuous-time Markov processIn probability theory, a continuous-time Markov process is a stochastic process { X : t ≥ 0 } that satisfies the Markov property and takes values from a set called the state space...
explore this property in greater detail.
In a broader sense, if a stochastic process of
random variableIn mathematics, random variables are used in the study of probability. They were developed to assist in the analysis of games of chance, stochastic events, and the results of scientific experiments by capturing only the mathematical properties necessary to answer probabilistic questions...
s determining a set of probabilities which can be factored in such a way that the
Markov property is obtained, then that process is said to have the
Markov-type property ; this is defined in detail below. Useful in applied research, members of such classes defined by their mathematics or area of application are referred to as Markov random fields, and occur in a number of situations, such as the
Ising modelThe Ising model, named after the physicist Ernst Ising, is a mathematical model in statistical mechanics. It has since been used to model diverse phenomena in which bits of information, interacting in pairs, produce collectiveeffects.-Definition :...
. The Markov property is named after
Andrey MarkovAndrey Andreyevich Markov was a Russian mathematician. He is best known for his work on theory of stochastic processes...
.
Definition
If one has a system composed of a set of
random variableIn mathematics, random variables are used in the study of probability. They were developed to assist in the analysis of games of chance, stochastic events, and the results of scientific experiments by capturing only the mathematical properties necessary to answer probabilistic questions...
s , then in general, the probability of a given random variable being in a state is written as
That is, in general, the probability of being in a state depends on the values of all of the other random variables . If, instead, one has that this probability only depends on
some, but not all of these, then one says that the collection has the
Markov property. Letting denote the subset of on which depends, one then writes this limited dependence as
Any collection of random variables having this property is referred to as a
Markov networkA Markov random field, Markov network or undirected graphical model is a graphical model in which a set of random variables have a Markov property described by an undirected graph. A Markov random field is similar to a Bayesian network in its representation of dependencies...
. The set is sometimes referred to as the
neighbors of ; alternately, it is the
Markov blanketIn machine learning, the Markov blanket for a node in a Bayesian network is the set of nodes composed of 's parents, its children, and its children's other parents. In a Markov network, the Markov blanket of a node is its set of neighbouring nodes...
of .
The probability distribution of a Markov network can always be written as a Gibbs distribution, that is, as
for an appropriate energy function
E defined on the subset . The
normalizing constantThe concept of a normalizing constant arises in probability theory and a variety of other areas of mathematics.-Definition and examples:In probability theory, a normalizing constant is a constant by which an everywhere non-negative function must be multiplied so the area under its graph is 1, e.g.,...
is known as the
partition functionThe partition function or configuration integral, as used in probability theory, information science and dynamical systems, is an abstraction of the definition of a partition function in statistical mechanics. It is a special case of a normalizing constant in probability theory, for the Boltzmann...
.
Markov networks are commonly seen in maximum entropy methods, since the Gibbs measure also has the property of being the unique stochastic measure that maximizes the entropy for a given energy functional.