Statistical independence

	Email		Print
	Bookmark		Link

Statistical independence

In probability theory

Probability theory

Probability theory is the branch of mathematics concerned with analysis of Statistical randomness phenomena. The central objects of probability theory are random variables, stochastic processes, and event s: mathematical abstractions of determinism events or measured quantities that may either be single occurrences or evolve over time in an a...
, to say that two event

Event (probability theory)

In probability theory, an event is a Set of outcomes to which a probability is assigned. Typically, when the sample space is finite, any subset of the sample space is an event ....
s are independent intuitively means that the occurrence of one event makes it neither more nor less probable that the other occurs. For example:

Similarly, two random variable

Random variable

In mathematics, random variables are used in the study of Randomness and probability. They were developed to assist in the analysis of Game of chance, stochastic events, and the results of experiment by capturing only the mathematical properties necessary to answer probability questions....
s are independent if the conditional probability distribution of either given the observed value of the other is the same as if the other's value had not been observed. The concept of independence extends to dealing with collections of more than two events or random variables.

In some instances the term "independent" is replaced by "statistically independent", "marginally independent" or "absolutely independent"

>

Here A ∩ B is the intersection

Intersection (set theory)

In mathematics, the intersection of two Set A and B is the set that contains all elements of A that also belong to B , but no other elements....
of A and B, that is, it is the event that both events A and B occur.

More generally, any collection of events -- possibly more than just two of them -- are mutually independent if and only if for any finite subset A₁, ..., A_n of the collection we have

This is called the multiplication rule for independent events.

If two events A and B are independent, then the conditional probability

Conditional probability

Conditional probability is the probability of some event A, given the occurrence of some other event B. Conditional probability is written P, and is read "the probability of A, given B"....
of A given B is the same as the unconditional (or marginal) probability of A, that is,

There are at least two reasons why this statement is not taken to be the definition of independence: (1) the two events A and B do not play symmetrical roles in this statement, and (2) problems arise with this statement when events of probability 0 are involved.

The conditional probability of event A given B is given by

(so long as Pr(B) ≠ 0 )

The statement above is equivalent to

which is the standard definition given above.

Note that independence does not have the same meaning as it does in the vernacular

Vernacular

Discussion

Ask a question about 'Statistical independence'

Start a new discussion about 'Statistical independence'

Answer questions from other users

Full Discussion Forum

Encyclopedia

In probability theory

Probability theory

Event (probability theory)

The event of getting a 6 the first time a die is rolled and the event of getting a 6 the second time are independent.
By contrast, the event of getting a 6 the first time a die is rolled and the event that the sum of the numbers seen on the first and second trials is 8 are dependent.
If two cards are drawn with replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are independent.
By contrast, if two cards are drawn without replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are dependent.

Similarly, two random variable

Random variable

Independent events

The standard definition says:

Two events A and B are independent if and only if

If and only if

If and only if, in logic and fields that rely on it such as mathematics and philosophy, is a biconditional logical connective between statements....
Pr(A ∩ B) = Pr(A)Pr(B).

Here A ∩ B is the intersection

Intersection (set theory)

Conditional probability

Vernacular

Vernacular refers to the native language of a country or a locality. In general linguistics, it is used to describe local languages as opposed to Lingua franca, official standards or global languages....
. For example an event is independent of itself if and only if

That is, if its probability is one or zero. Thus if an event or its complement

Complement (set theory)

In discrete mathematics and predominantly in set theory, a complement is a concept used in comparisons of sets to refer to the unique values of one set in relation to another....
almost surely

Almost surely

In probability theory, one says that an event happens almost surely if it happens with probability one. The concept is analogous to the concept of "almost everywhere" in measure theory....
occurs, it is independent of itself. For example, if event A is choosing any number but 0.5 from a uniform distribution

Uniform distribution (continuous)

In probability theory and statistics, the continuous uniform distribution is a family of probability distributions such that for each member of the family, all interval s of the same length on the distribution's support are equally probable....
on the unit interval

Unit interval

In mathematics, the unit interval is the interval [0,1], that is, the set of all real numbers that are greater than or equal to 0 and less than or equal to 1....
, A is independent of itself, even though, tautologically

Tautology (logic)

In propositional logic, a tautology is a propositional formula that is true under any possible Valuation of its propositional variables. For example, the propositional formula is a tautology, because the statement is true for any valuation of A....
, A fully determines A.

Independent random variables

What is defined above is independence of events. In this section we treat independence of random variable

Random variable

Real number

In mathematics, the real numbers may be described informally in several different ways. The real numbers include both rational numbers, such as 42 and −23/129, and irrational numbers, such as pi and the square root of two; or, a real number can be given by an infinite decimal representation, such as 2.4871773339...., where the digits co...
-valued random variable and a is a number then the event is the set of outcomes that correspond to X being less than or equal to a. Since these are sets of outcomes that have probabilities, it makes sense to refer to events of this sort being independent of other events of this sort.

Two random variables X and Y are independent if and only if

If and only if

If and only if, in logic and fields that rely on it such as mathematics and philosophy, is a biconditional logical connective between statements....
for any numbers a and b the events (the outcomes where X being less than or equal to a) and are independent events as defined above. Similarly an arbitrary collection of random variables -- possible more than just two of them -- is independent precisely if for any finite collection X₁, ..., X_n and any finite set of numbers a₁, ..., a_n, the events ,..., are independent events as defined above.

The measure-theoretically inclined may prefer to substitute events for events in the above definition, where A is any Borel set

Borel algebra

In mathematics, the Borel algebra on a topological space X is a sigma-algebra of subsets of X associated with the topology of X. In the mathematics literature, there are at least two nonequivalent definitions of this σ-algebra:...
. That definition is exactly equivalent to the one above when the values of the random variables are real number

Real number

In mathematics, the real numbers may be described informally in several different ways. The real numbers include both rational numbers, such as 42 and −23/129, and irrational numbers, such as pi and the square root of two; or, a real number can be given by an infinite decimal representation, such as 2.4871773339...., where the digits co...
s. It has the advantage of working also for complex-valued random variables or for random variables taking values in any topological space

Topological space

Topological spaces are mathematical structures that allow the formal definition of concepts such as convergence, connected space, and Continuous function ....
.

If any two of a collection of random variables are independent, they may nonetheless fail to be mutually independent; this is called pairwise independence

Pairwise independence

In probability theory, a pairwise independent collection of random variables is a set of random variables any two of which are statistical independence....
.

If X and Y are independent, then the expectation operator

Expected value

In probability theory and statistics, the expected value of a random variable is the Lebesgue integral of the random variable with respect to its probability measure....
E has the nice property

E[X Y] = E[X] E[Y],

and for the variance

Variance

In probability theory and statistics, the variance of a random variable, probability distribution, or sample is one measure of statistical dispersion, averaging the squared distance of its possible values from the expected value ....
we have

var(X + Y) = var(X) + var(Y),

so the covariance

Covariance

In probability theory and statistics, covariance is a measure of how much two variables change together .If two variables tend to vary together , then the covariance between the two variables will be positive....
cov(X,Y) is zero. (The converse of these, i.e. the proposition that if two random variables have a covariance of 0 they must be independent, is not true. See uncorrelated

Uncorrelated

In probability theory and statistics, two real-valued random variables are said to be uncorrelated if their covariance is zero.Uncorrelated random variables have a correlation of zero, except in the trivial case when both variables have variance zero ....
.)

Furthermore, random variables X and Y with distribution function

Cumulative distribution function

In probability theory and statistics, the cumulative distribution function or just distribution function, completely describes the probability distribution of a real-valued random variable X....
s F_X(x) and F_Y(y), and probability densities

Probability density function

In mathematics, a probability density function is a function that represents a probability distribution in terms of integrals.Formally, a probability distribution has density ƒ, if ƒ is a non-negative Lebesgue integration function such that the probability of the interval [a, b] is given by...
f_X(x) and f_Y(y), are independent if and only if the combined random variable (X,Y) has a joint distribution

or equivalently, a joint density

Similar expressions characterise independence more generally for more than two random variables.

Conditionally independent random variables

Intuitively, two random variables X and Y are conditionally independent given Z if, once Z is known, the value of Y does not add any additional information about X. For instance, two measurements X and Y of the same underlying quantity Z are not independent, but they are conditionally independent given Z (unless the errors in the two measurements are somehow connected).

The formal definition of conditional independence is based on the idea of conditional distribution
Conditional distribution

Given two jointly distributed random variables X and Y, the conditional probability distribution of Y given X is the probability distribution of Y when X is known to be a particular value....
s. If X, Y, and Z are discrete random variables, then we define X and Y to be conditionally independent given Z if

for all x, y and z such that . On the other hand, if the random variables are continuous and have a joint probability density function
Probability density function

In mathematics, a probability density function is a function that represents a probability distribution in terms of integrals.Formally, a probability distribution has density ƒ, if ƒ is a non-negative Lebesgue integration function such that the probability of the interval [a, b] is given by...
p, then X and Y are conditionally independent given Z if

for all real numbers x, y and z such that .

If X and Y are conditionally independent given Z, then for any x, y and z with . That is, the conditional distribution for X given Y and Z is the same as that given Z alone. A similar equation holds for the conditional probability density functions in the continuous case.

Independence can be seen as a special kind of conditional independence, since probability can be seen as a kind of conditional probability given no events.

External links

, a provider of "true" random numbers, with randomness coming from atmospheric noises.

Statistical independence

Independent events

Independent random variables

Conditionally independent random variables

See also

External links