Total variation distance of probability measures
Encyclopedia
In probability theory
, the total variation distance between two probability measure
s P and Q on a sigma-algebra
F is
Informally, this is the largest possible difference between the probabilities that the two probability distribution
s can assign to the same event.
For a finite alphabet
we can write
Sometimes the statistical distance
between two probability distributions is also defined without the division by two.
Probability theory
Probability theory is the branch of mathematics concerned with analysis of random phenomena. The central objects of probability theory are random variables, stochastic processes, and events: mathematical abstractions of non-deterministic events or measured quantities that may either be single...
, the total variation distance between two probability measure
Probability measure
In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies measure properties such as countable additivity...
s P and Q on a sigma-algebra
Sigma-algebra
In mathematics, a σ-algebra is a technical concept for a collection of sets satisfying certain properties. The main use of σ-algebras is in the definition of measures; specifically, the collection of sets over which a measure is defined is a σ-algebra...
F is
Informally, this is the largest possible difference between the probabilities that the two probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....
s can assign to the same event.
For a finite alphabet
Categorical distribution
In probability theory and statistics, a categorical distribution is a probability distribution that describes the result of a random event that can take on one of K possible outcomes, with the probability of each outcome separately specified...
we can write
Sometimes the statistical distance
Statistical distance
In statistics, probability theory, and information theory, a statistical distance quantifies the distance between two statistical objects, which can be two samples, two random variables, or two probability distributions, for example.-Metrics:...
between two probability distributions is also defined without the division by two.