Total variation - AbsoluteAstronomy.com

Mathematics

Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...

, the total variation identifies several slightly different concepts, related to the (local

Local property

In mathematics, a phenomenon is sometimes said to occur locally if, roughly speaking, it occurs on sufficiently small or arbitrarily small neighborhoods of points.-Properties of a single space:...

or global) structure of the codomain

Codomain

In mathematics, the codomain or target set of a function is the set into which all of the output of the function is constrained to fall. It is the set in the notation...

of a function

Function (mathematics)

In mathematics, a function associates one quantity, the argument of the function, also known as the input, with another quantity, the value of the function, also known as the output. A function assigns exactly one output to each input. The argument and the value may be real numbers, but they can...

or a measure

Measure

- Legal :* Measure of the Church of England is a law passed by the General Synod and the UK Parliament equivalent of an Act* Measure of the National Assembly for Wales, a law specific to Wales passed by the Welsh Assembly between 2007 and 2011...

. For a real-valued

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

continuous function

Continuous function

In mathematics, a continuous function is a function for which, intuitively, "small" changes in the input result in "small" changes in the output. Otherwise, a function is said to be "discontinuous". A continuous function with a continuous inverse function is called "bicontinuous".Continuity of...

, defined on an interval

Interval (mathematics)

In mathematics, a interval is a set of real numbers with the property that any number that lies between two numbers in the set is also included in the set. For example, the set of all numbers satisfying is an interval which contains and , as well as all numbers between them...

[a, b] ⊂ ℝ, its total variation on the interval of definition is a measure of the one-dimensional arclength of the curve with parametric equation x →

(x), for x ∈ [a,b].

Historical notice

The concept of total variation for functions of one real variable was first introduced by Camille Jordan

Camille Jordan

Marie Ennemond Camille Jordan was a French mathematician, known both for his foundational work in group theory and for his influential Cours d'analyse. He was born in Lyon and educated at the École polytechnique...

in the paper . He used the new concept in order to prove a convergence theorem for Fourier series

Fourier series

In mathematics, a Fourier series decomposes periodic functions or periodic signals into the sum of a set of simple oscillating functions, namely sines and cosines...

of discontinuous periodic function

Periodic function

In mathematics, a periodic function is a function that repeats its values in regular intervals or periods. The most important examples are the trigonometric functions, which repeat over intervals of length 2π radians. Periodic functions are used throughout science to describe oscillations,...

s whose variation is bounded

Bounded variation

In mathematical analysis, a function of bounded variation, also known as a BV function, is a real-valued function whose total variation is bounded : the graph of a function having this property is well behaved in a precise sense...

. The extension of the concept to functions of more than one variable however is not simple for some reasons.

Total variation for functions of one real variable

The total variation of a real

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

-valued (or more generally complex

Complex number

A complex number is a number consisting of a real part and an imaginary part. Complex numbers extend the idea of the one-dimensional number line to the two-dimensional complex plane by using the number line for the real part and adding a vertical axis to plot the imaginary part...

-valued) function

Function (mathematics)

, defined on an interval

Interval (mathematics)

⊂ℝ is the quantity

where the supremum

Supremum

In mathematics, given a subset S of a totally or partially ordered set T, the supremum of S, if it exists, is the least element of T that is greater than or equal to every element of S. Consequently, the supremum is also referred to as the least upper bound . If the supremum exists, it is unique...

runs over the set of all partitions

Partition of an interval

In mathematics, a partition, P of an interval [a, b] on the real line is a finite sequence of the formIn mathematics, a partition, P of an interval [a, b] on the real line is a finite sequence of the form...

of the given interval

Interval (mathematics)

Total variation for functions of n>1 real variables

Let be an open subset of ℝⁿ. Given a function belonging to , the total variation of in is defined as

where

is the set of continuously differentiable

Smooth function

In mathematical analysis, a differentiability class is a classification of functions according to the properties of their derivatives. Higher order differentiability classes correspond to the existence of more derivatives. Functions that have derivatives of all orders are called smooth.Most of...

vector functions of compact support contained in

, and

is the essential supremum norm

Norm (mathematics)

In linear algebra, functional analysis and related areas of mathematics, a norm is a function that assigns a strictly positive length or size to all vectors in a vector space, other than the zero vector...

. Note that this definition does not require that the domain

⊆ℝⁿ of the given function is a bounded set

Bounded set

In mathematical analysis and related areas of mathematics, a set is called bounded, if it is, in a certain sense, of finite size. Conversely, a set which is not bounded is called unbounded...

Total variation in measure theory

Following , consider a signed measure

Signed measure

In mathematics, signed measure is a generalization of the concept of measure by allowing it to have negative values. Some authors may call it a charge, by analogy with electric charge, which is a familiar distribution that takes on positive and negative values.-Definition:There are two slightly...

on a measurable space

Sigma-algebra

In mathematics, a σ-algebra is a technical concept for a collection of sets satisfying certain properties. The main use of σ-algebras is in the definition of measures; specifically, the collection of sets over which a measure is defined is a σ-algebra...

: then it is possible to define two set function

Set function

In mathematics, a set function is a function whose input is a set. The output is usually a number. Often the input is a set of real numbers, a set of points in Euclidean space, or a set of points in some measure space.- Examples :...

and

, respectively called upper variation and lower variation, as follows

clearly

The variation (also called absolute variation) of the signed measure is the set function

and its total variation is defined as the value of this measure on the whole space of definition, i.e.

uses upper and lower variations to prove the Hahn–Jordan decomposition

Hahn decomposition theorem

In mathematics, the Hahn decomposition theorem, named after the Austrian mathematician Hans Hahn, states that given a measurable space and a signed measure μ defined on the σ-algebra Σ, there exist two sets P and N in Σ such that:...

: according to his version of this theorem, the upper and lower variation are respectively a non-negative and a non-positive measure

Measure (mathematics)

In mathematical analysis, a measure on a set is a systematic way to assign to each suitable subset a number, intuitively interpreted as the size of the subset. In this sense, a measure is a generalization of the concepts of length, area, and volume...

. Using a more modern notation, define

Then and are two non-negative measure

Measure (mathematics)

s such that

The last measure is sometimes called, by abuse of notation

Abuse of notation

In mathematics, abuse of notation occurs when an author uses a mathematical notation in a way that is not formally correct but that seems likely to simplify the exposition or suggest the correct intuition . Abuse of notation should be contrasted with misuse of notation, which should be avoided...

, total variation measure.

If the measure is complex-valued

Complex number

i.e. is a complex measure

Complex measure

In mathematics, specifically measure theory, a complex measure generalizes the concept of measure by letting it have complex values. In other words, one allows for sets whose size is a complex number.-Definition:...

, its upper and lower variation cannot be defined and the Hahn–Jordan decomposition theorem can only be applied to its real and imaginary parts. However, it is possible to follow and define the total variation of the complex-valued measure as follows

The variation of the complex-valued measure is the set function

Set function

where the supremum

Supremum

is taken over all partitions of a measurable set into a finite number of disjoint measurable subsets.

The variation so defined is a positive measure (see ) and coincides with the one defined by when is a signed measure

Signed measure

: its total variation is defined as above. This definition works also if is a vector measure

Vector measure

In mathematics, a vector measure is a function defined on a family of sets and taking vector values satisfying certain properties. It is a generalization of the concept of measure, which takes nonnegative real values only.-Definitions and first consequences:...

: the variation is then defined by the following formula

where the supremum is as above. Note also that this definition is slightly more general than the one given by since it requires only to consider finite partitions of the space : this implies that it can be used also to define the total variation on finitely-additive measures.

Total variation of probability measures

The total variation of any probability measure

Probability measure

In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies measure properties such as countable additivity...

is exactly one, therefore it is not interesting as a means of investigating the properties of such measures. However, when μ and ν are probability measure

Probability measure

In mathematics, a probability measure is a real-valued function defined on a set of events in a probability space that satisfies measure properties such as countable additivity...

, the total variation distance of probability measures can be defined as

and its values are non-trivial. Informally, this is the largest possible difference between the probabilities that the two probability distribution

Probability distribution

In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....

s can assign to the same event. For a categorical distribution

Categorical distribution

In probability theory and statistics, a categorical distribution is a probability distribution that describes the result of a random event that can take on one of K possible outcomes, with the probability of each outcome separately specified...

it is possible to write the total variation distance as follows

The total variational distance for categorical probability distributions is called statistical distance

Statistical distance

In statistics, probability theory, and information theory, a statistical distance quantifies the distance between two statistical objects, which can be two samples, two random variables, or two probability distributions, for example.-Metrics:...

: sometimes, in the definition of this distance, the factor

is omitted.

Total variation of differentiable functions

The total variation of a differentiable function

Differentiable function

In calculus , a differentiable function is a function whose derivative exists at each point in its domain. The graph of a differentiable function must have a non-vertical tangent line at each point in its domain...

can be expressed as an integral

Integral

Integration is an important concept in mathematics and, together with its inverse, differentiation, is one of the two main operations in calculus...

involving the given function instead of as the supremum

Supremum

of the functional

Functional

Generally, functional refers to something able to fulfill its purpose or function.*Functionalism and Functional form, movements in architectural design*Functional group, certain atomic combinations that occur in various molecules, e.g...

s of definitions and .

The form of the total variation of a differentiable functions of one variable

The total variation of a differentiable function

Differentiable function

, defined on an interval

Interval (mathematics)

⊂ℝ, has the following expression if f' is Riemann integrable

The form of the total variation of a differentiable functions of several variables

Given a differentiable function

Differentiable function

defined on a bounded

Bounded set

In mathematical analysis and related areas of mathematics, a set is called bounded, if it is, in a certain sense, of finite size. Conversely, a set which is not bounded is called unbounded...

open set

Open set

The concept of an open set is fundamental to many areas of mathematics, especially point-set topology and metric topology. Intuitively speaking, a set U is open if any point x in U can be "moved" a small amount in any direction and still be in the set U...

⊆ℝⁿ, the total variation of has the following expression

Proof

The first step in the proof is to first prove an equality which follows from the Gauss-Ostrogradsky theorem.

Lemma

Under the conditions of the theorem, the following equality holds:

Proof of the lemma

From the Gauss-Ostrogradsky theorem:

by subtituting

, we have:

where

is zero on the border of

by definition:

Proof of the equality

Under the conditions of the theorem, from the lemma we have:

in the last part

could be omitted, because by definition it's considerate supremum is at most one.

On the other hand we consider

and

which is the up to

approximation of

with the same integral. We can do this hence

is dense in

. Now again substituting into the lemma:

This means we have a convergent sequence of

that tends to

as well as we know that

. q.e.d.

It can be seen from the proof that the supremum is attained when

The function

Function (mathematics)

is said to be of bounded variation

Bounded variation

precisely if its total variation is finite.

Total variation of a measure

The total variation is a norm

Norm (mathematics)

defined on the space of measures of bounded variation. The space of measures on a σ-algebra of sets is a Banach space

Banach space

In mathematics, Banach spaces is the name for complete normed vector spaces, one of the central objects of study in functional analysis. A complete normed vector space is a vector space V with a norm ||·|| such that every Cauchy sequence in V has a limit in V In mathematics, Banach spaces is the...

, called the ca space, relative to this norm. It is contained in the larger Banach space, called the ba space

Ba space

In mathematics, the ba space ba of an algebra of sets \Sigma is the Banach space consisting of all bounded and finitely additive measures on \Sigma. The norm is defined as the variation, that is \|\nu\|=|\nu|....

, consisting of finitely additive (as opposed to countably additive) measures, also with the same norm. The distance function associated to the norm gives rise to the total variation distance between two measures μ and ν.

For finite measures on ℝ, the link between the total variation of a measure μ and the total variation of a function, as described above, goes as follows. Given μ, define a function

Then, the total variation of the signed measure μ is equal to the total variation, in the above sense, of the function φ. In general, the total variation of a signed measure can be defined using Jordan's decomposition theorem

Hahn decomposition theorem

for any signed measure μ on a measurable space

Applications

Total variation can be seen as a non-negative real

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

-valued functional

Functional (mathematics)

In mathematics, and particularly in functional analysis, a functional is a map from a vector space into its underlying scalar field. In other words, it is a function that takes a vector as its input argument, and returns a scalar...

defined on the space of real-valued

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

function

Function (mathematics)

s (for the case of functions of one variable) or on the space of integrable functions (for the case of functions of several variables). As a functional, total variation finds applications in several branches of mathematics and engineering, like optimal control

Optimal control

Optimal control theory, an extension of the calculus of variations, is a mathematical optimization method for deriving control policies. The method is largely due to the work of Lev Pontryagin and his collaborators in the Soviet Union and Richard Bellman in the United States.-General method:Optimal...

, numerical analysis

Numerical analysis

Numerical analysis is the study of algorithms that use numerical approximation for the problems of mathematical analysis ....

, and calculus of variations

Calculus of variations

Calculus of variations is a field of mathematics that deals with extremizing functionals, as opposed to ordinary calculus which deals with functions. A functional is usually a mapping from a set of functions to the real numbers. Functionals are often formed as definite integrals involving unknown...

, where the solution to a certain problem has to minimize

Maxima and minima

In mathematics, the maximum and minimum of a function, known collectively as extrema , are the largest and smallest value that the function takes at a point either within a given neighborhood or on the function domain in its entirety .More generally, the...

its value. As an example, use of the total variation functional is common in the following two kind of problems

Numerical analysis of differential equations: it is the science of finding approximate solutions to differential equation
Differential equation
A differential equation is a mathematical equation for an unknown function of one or several variables that relates the values of the function itself and its derivatives of various orders...

s. Applications of total variation to this problems are detailed in the article "total variation diminishing
Total variation diminishing
In numerical methods, total variation diminishing is a property of certain discretization schemes used to solve hyperbolic partial differential equations...

"

Image denoising: in image processing
Image processing
In electrical engineering and computer science, image processing is any form of signal processing for which the input is an image, such as a photograph or video frame; the output of image processing may be either an image or, a set of characteristics or parameters related to the image...

, denoising is a collection of methods used to reduce the noise
Electronic noise
Electronic noise is a random fluctuation in an electrical signal, a characteristic of all electronic circuits. Noise generated by electronic devices varies greatly, as it can be produced by several different effects...

in an image
Image
An image is an artifact, for example a two-dimensional picture, that has a similar appearance to some subject—usually a physical object or a person.-Characteristics:...

reconstructed from data obtained by electronic means, for example data transmission
Data transmission
Data transmission, digital transmission, or digital communications is the physical transfer of data over a point-to-point or point-to-multipoint communication channel. Examples of such channels are copper wires, optical fibres, wireless communication channels, and storage media...

or sensing
Sensor
A sensor is a device that measures a physical quantity and converts it into a signal which can be read by an observer or by an instrument. For example, a mercury-in-glass thermometer converts the measured temperature into expansion and contraction of a liquid which can be read on a calibrated...

. Total variation denoising
Total variation denoising
In signal processing, Total variation denoising, also known as total variation regularization is a process, most often used in digital image processing that has applications in noise removal. It is based on the principle that signals with excessive and possibly spurious detail have high total...

is the name for the application of total variation to image noise reduction; further details can be found in the paper .

Historical notice

Total variation for functions of one real variable

Total variation for functions of n>1 real variables

Total variation in measure theory

Total variation of probability measures

Total variation of differentiable functions

The form of the total variation of a differentiable functions of one variable

The form of the total variation of a differentiable functions of several variables

Proof

Lemma

Proof of the lemma

Proof of the equality

Total variation of a measure

Applications

See also

Theory

Applications